Hello and welcome to our community! Is this your first visit?
Register
Enjoy an ad free experience by logging in. Not a member yet? Register.
Results 1 to 3 of 3

Thread: Robots

  1. #1
    New to the CF scene
    Join Date
    Jul 2007
    Location
    Wales
    Posts
    8
    Thanks
    0
    Thanked 0 Times in 0 Posts

    Robots

    For my website I have included the code for robots and googlebot and I would like to know if there are any other robots?

  • #2
    The fat guy next door VIPStephan's Avatar
    Join Date
    Jan 2006
    Location
    Halle (Saale), Germany
    Posts
    8,478
    Thanks
    6
    Thanked 981 Times in 954 Posts
    There are a lot of robots. Refer to Wikipediaís robots.txt to see how many of them are adressed. However, many of them also have no impact on your site.

    Also the meta tags for robots donít have any impact on your siteís indexing or rating. Use a robots.txt file to exclude robots but otherwise your page will be indexed anyway. And ranking depends on the quality of your content and how many links to your site from other trusted sites there are. Those meta tags are nearly irrelevant.

  • #3
    ess
    ess is offline
    Regular Coder
    Join Date
    Oct 2006
    Location
    United Kingdom
    Posts
    865
    Thanks
    7
    Thanked 29 Times in 28 Posts
    be careful when you use robots.txt files. Some websites would recommend that you include directories and/or files that you don't want web crawlers from accessing.....which obviously informs script kiddies and wanna be hackers...that these folders/files may contain information that might help them in hacking into your website.

    Personally, i don't use them at all....as I find them to be a cause for concern more than a helpful solution that supposedly should affect website ranking...which it doesn't. All it suppose to do is to tell web crawlers which files are supposed to be indexed and which are supposed to remove whilst crawling your website etc.


  •  

    Posting Permissions

    • You may not post new threads
    • You may not post replies
    • You may not post attachments
    • You may not edit your posts
    •