Hello and welcome to our community! Is this your first visit?
Register
Enjoy an ad free experience by logging in. Not a member yet? Register.
Results 1 to 6 of 6
  1. #1
    Regular Coder
    Join Date
    Dec 2006
    Posts
    183
    Thanks
    17
    Thanked 0 Times in 0 Posts

    Avoid Page Indexing by SE's but Allow Other Links to be Indexed

    I have a url like

    Code:
    http://mysite.com/products/
    this is page one. The page 2 url is like this

    Code:
    http://mysite.com/products/pg2/
    My question is how can I avoid having the http://mysite.com/parameter/pg2/ page indexed by google or the other search engines but have the links on that page 2 page followed and indexed?

    Thanks

  • #2
    Senior Coder gnomeontherun's Avatar
    Join Date
    Sep 2007
    Location
    Houston
    Posts
    2,846
    Thanks
    10
    Thanked 238 Times in 229 Posts
    You need a robots.txt file placed at the root of your website. Inside put

    Code:
    User-agent: *
    Disallow: /parameter/pg2/
    http://www.robotstxt.org/

  • #3
    Senior Coder
    Join Date
    Jul 2005
    Location
    UK
    Posts
    1,051
    Thanks
    6
    Thanked 13 Times in 13 Posts
    You need a robots.txt file placed at the root of your website. Inside put

    Code:

    User-agent: *
    Disallow: /parameter/pg2/

    http://www.robotstxt.org/
    This will not allow the links from page 2 to be indexed like the OP wants.

    Try:

    Code:
    <meta name="robots" content="noarchive, follow, noindex" />

  • Users who have thanked Pennimus for this post:

    thekevin07 (04-24-2008)

  • #4
    Senior Coder gnomeontherun's Avatar
    Join Date
    Sep 2007
    Location
    Houston
    Posts
    2,846
    Thanks
    10
    Thanked 238 Times in 229 Posts
    Yes true, I misread the post.

  • #5
    New Coder
    Join Date
    Apr 2008
    Location
    Kansas City, KS
    Posts
    13
    Thanks
    0
    Thanked 0 Times in 0 Posts
    Well I was going to say to add a simple nofollow buy you beat me to it Pennimus. lol

  • #6
    Regular Coder
    Join Date
    Dec 2006
    Posts
    183
    Thanks
    17
    Thanked 0 Times in 0 Posts
    Quote Originally Posted by Pennimus View Post
    This will not allow the links from page 2 to be indexed like the OP wants.

    Try:

    Code:
    <meta name="robots" content="noarchive, follow, noindex" />
    Perfect thanks


  •  

    Posting Permissions

    • You may not post new threads
    • You may not post replies
    • You may not post attachments
    • You may not edit your posts
    •