Disallow folder indexing

[dan-dan]

I'm creating a site locally, though I want to test it live.

I don't have any hosting at the moment, though I'm able to host it on a friends site I created for her some time ago.

However, I don't want the folder and its contents to get indexed by any search engines.

I read that I can use the below rule within robots.txt

User-agent: *
Disallow: /

My question is, will this work being placed within a specific folder, or is it something that only works from root for the entire site.

If the later, is there a way to do it?

Thank you.

[ShaneC]

Some search engines will look for it in each directory, but not all. As such, in order for it to be effective, you need to put the file in the "root" directory (i.e: the top level directory, http://example.org/robots.txt).

Having said this, you can tell robots to simply not index your directory. So, instead of doing Disallow: /, which would tell search engines to not index the entire site, replace that with Disallow: /my_directory/ which will restrict no indexing to your directory (and sub-directories thereof).

For more information on this, check out this site.

[dan-dan]

Thanks for the explanation and link ShaneC, much appreciated.

[Whisper]

does this .htaccess file go in the folder of the "Disallowed" listing or in the root of the site?

[MarkR]

I prefer to just use .htaccess and add a password to the directory.

That way search engines can't access it (some shady web crawlers ignore robots.txt rules) and it means people can't access it by accident.