PDA

View Full Version : htaccess - Can it identify Search Engine Spiders?


BatCountry
07-13-2009, 08:30 PM
Just wondering whether htaccess can identify or sniff out search engine spiders like msn/bing, yahoo and google?

I long ago switched domains on a site I have and still have people accessing the site from one domain, I want to make it so that if a person enters in that url they can still see the page but if a spiderbot like google visits it will show a 301 redirect to the new domain.

How would I go about doing that with htaccess?

Inigoesdr
07-14-2009, 12:32 AM
You could accomplish this by matching the %{HTTP_USER_AGENT} string with mod_rewrite and then performing the redirect. Google for some mod_rewrite tutorials if you don't know how to use it.

primefalcon
07-16-2009, 05:14 AM
you could also just do it just using php by using the $_SERVER['HTTP_USER_AGENT'] preset variable

it career
07-16-2009, 07:56 AM
Yes you can do it but that is a sure way of getting your website penalized in SERP.

BatCountry
07-24-2009, 02:26 AM
well it's the same content that's being shown, only I'm controlling the urls.

I was previously using my hosts tmp subdomain so:
http://subdomain.webhost.org

and I want to now use my domain for seo purposes. I want the old url to still work for users, but for Google to see a 301 redirect that uses the domain url instead (same content, just different url).

I was thinking of doing it with php, but the way my site is if someone access an independent pdf or html file it would be impossible or difficult to get that to redirect as well. Basically I was going to sniff the url being used, if it detects googlebot 301 redirect to the url with the domain, otherwise if no googlebot just show the regular url with a notice to the user to use the domain instead.

Anyone tell me how to go about doing that with htaccess if possible?