CodingForums.com - View Single Post - Rewrite robots.txt with robots.php, but block direct access to robots.php itself?

Rowsdower! · 11-27-2012, 04:39 PM

I have the following as part of my .htaccess file:

Code:

RewriteRule ^(.*)robots.txt$ $1/robots.php [L]

This allows me to do some PHP stuff with page metrics before actually sending the robots.txt directives, and it makes the presentation of robots.txt seemless as expected.

However, today Google threw me a curve ball by requesting robots.php out of the blue. I don't want them (or anyone else) to do that so I need to block direct access to robots.php, issuing a 404. To do that, I tried the following:

Code:

RedirectMatch 404 ^(.*)robots.php$

And it worked...too well. Now my robots.txt returns my 404 page as well.

I presume this is a direct result of the rewrite rule which must count just the same as a direct URI "get" request to the page (which I had not intuited).

So is there a way to simultaneously serve robots.php when robots.txt is requested AND issue a 404 error when robots.php is directly requested?

11-27-2012, 04:39 PM	PM User \| #1
Rowsdower! Senior Coder Join Date: Oct 2008 Location: Some say it's everything. Posts: 2,007 Thanks: 5 Thanked 395 Times in 388 Posts	Rewrite robots.txt with robots.php, but block direct access to robots.php itself? I have the following as part of my .htaccess file: Code: RewriteRule ^(.)robots.txt$ $1/robots.php [L] This allows me to do some PHP stuff with page metrics before actually sending the robots.txt directives, and it makes the presentation of robots.txt seemless as expected. However, today Google threw me a curve ball by requesting robots.php out of the blue. I don't want them (or anyone else) to do that so I need to block direct access to robots.php, issuing a 404. To do that, I tried the following: Code: RedirectMatch 404 ^(.)robots.php$ And it worked...too well. Now my robots.txt returns my 404 page as well. I presume this is a direct result of the rewrite rule which must count just the same as a direct URI "get" request to the page (which I had not intuited). So is there a way to simultaneously serve robots.php when robots.txt is requested AND issue a 404 error when robots.php is directly requested? __________________ The object of opening the mind, as of opening the mouth, is to shut it again on something solid. –G.K. Chesterton See Mediocrity in its Infancy It's usually a good idea to start out with this at the VERY TOP of your CSS: *` {border:0;margin:0;padding:0;}` Seek and you shall find... basically: validate your markup \| view your page cross-browser/cross-platform \| free web tutorials \| free hosting**