Hello and welcome to our community! Is this your first visit?
Register
Enjoy an ad free experience by logging in. Not a member yet? Register.
Results 1 to 3 of 3
  1. #1
    New to the CF scene
    Join Date
    Apr 2013
    Posts
    1
    Thanks
    0
    Thanked 0 Times in 0 Posts

    Problems with robots.txt and Google.

    Hi everyone,

    I am having a bit of a problem. Whenever I type my website into a google search engine, I get the following:

    "A description for this result is not available because of this site's robots.txt – learn more."

    This was a little surprising to me since I didn't originally upload a robots.txt file, or even know what they were. So I uploaded one to try and negate whatever was going on. I wanted it to stop blocking my site in google. This is what I typed and uploaded.

    User-agent: *
    Allow: /

    Still no avail, though. I'm not sure if it's something I'm doing or if Google's crawlers just haven't re-crawled. Does anyone have any idea what could be going on? Has anyone else ever had this problem? How did you solve it?

  • #2
    Regular Coder patryk's Avatar
    Join Date
    Oct 2012
    Location
    /dev/couch
    Posts
    398
    Thanks
    2
    Thanked 64 Times in 64 Posts
    are you registered here: https://www.google.com/webmasters/ ?
    if no, set up an acount, register your site and tell it to re-fetch your robots.txt and rest of documents.
    it sometimes takes several weeks for resoults to show in google though

    -------------------------------------------------------------------------------
    "Real Programmers can write assembly code in any language" - Larry Wall

  • #3
    New to the CF scene
    Join Date
    Jul 2012
    Posts
    1
    Thanks
    0
    Thanked 0 Times in 0 Posts
    Maybe you have to check the robots via robots.txt checker, for example via http://tool.motoricerca.info/robots-checker.phtml. Try to submit robots.txt file and see what the problem.


  •  

    Tags for this Thread

    Posting Permissions

    • You may not post new threads
    • You may not post replies
    • You may not post attachments
    • You may not edit your posts
    •