Hello and welcome to our community! Is this your first visit?
Register
Enjoy an ad free experience by logging in. Not a member yet? Register.
Results 1 to 5 of 5
  1. #1
    Senior Coder doubledee's Avatar
    Join Date
    Mar 2011
    Location
    Arizona
    Posts
    1,057
    Thanks
    25
    Thanked 0 Times in 0 Posts

    What can Google crawl?

    What exactly can Google spiders see on any given webpage?


    -------------
    For example, I have a drop-down box like this...
    PHP Code:
        <!-- Sort Menu -->
        <
    div class="drop">
            <
    label for="sortrequest">Sort by:</label>
            <
    select name="sortrequest">
                <
    option value="by-date-desc">Date (Newest 1st)</option>
                <
    option value="by-date-asc">Date (Oldest 1st)</option>
            </
    select>
                                        
            <!-- 
    Submit Button -->
            <
    input type='submit' name='submit' value='Go'/>
        </
    div><!-- End of SORT MENU --> 
    Is Google able to peak inside my $_GET variable?

    And is Google able to crawl that temporary URL?


    -------------
    And when that Form is submitted, it temporarily populates the $_GET variable with a value and produces a URL like this...
    Code:
    local.debbie/finance/markets/yellen-promises-to-keep-rates-low?sortrequest=by-title-desc&submit=Go
    Is Google able to crawl that?


    -------------
    And then my PHP script grabs that temporary URL and does a redirect like this...
    PHP Code:
        // Check for Sort-Request in URL.
        
    if (isset($_GET['sortrequest']) && $_GET['sortrequest']){
            
    // Sort-Request Found in URL.
            // Set Case.
            
    $sortRequest strtolower($_GET['sortrequest']);
            
            
    // Determine Sort-Type.
        
    switch ($sortRequest){
          case 
    "by-date-desc":
            
    header("Location: " BASE_URL "/$sectionSlug/$subsectionSlug/?sortname=by-date&sortdir=desc&page=1");
            break;
                
          case 
    "by-date-asc":
            
    header("Location: " BASE_URL "/$sectionSlug/$subsectionSlug/?sortname=by-date&sortdir=asc&page=1");
            break;
        
            and 
    so on... 
    Is Google able to crawl that?


    -------------
    I ask these questions, because I am minimize having "Duplicate Content" with Google as far as sorting goes.

    Make sense?

    Sincerely,


    Debbie

  • #2
    Regular Coder Arcticwarrio's Avatar
    Join Date
    May 2012
    Location
    UK
    Posts
    721
    Thanks
    20
    Thanked 84 Times in 84 Posts
    put this in your head <meta name="robots" content="noindex">

    or but a txt file in the root:

    https://developers.google.com/webmas...ocs/robots_txt
    Last edited by Arcticwarrio; 05-07-2014 at 10:36 PM.
    There are 10 types of people on CodingForums,
    Those who understand Binary and those who dont.
    Get Cloud Hosting now from only£59 / month

  • #3
    Senior Coder doubledee's Avatar
    Join Date
    Mar 2011
    Location
    Arizona
    Posts
    1,057
    Thanks
    25
    Thanked 0 Times in 0 Posts
    Quote Originally Posted by Arcticwarrio View Post
    put this in your head <meta name="robots" content="noindex">

    or but a txt file in the root:

    https://developers.google.com/webmas...ocs/robots_txt
    Sorry, but that doesn't answer my questions...


    Debbie

  • #4
    Regular Coder
    Join Date
    Oct 2004
    Posts
    356
    Thanks
    0
    Thanked 16 Times in 16 Posts
    Yes to crawling <form>s: Official Google Webmaster Central Blog: Crawling through HTML forms (from 2008).

    You should make the redirects permanent {header()}
    PHP Code:
    <?php

    // 301 Moved Permanently
    header("Location: http://example.com/foo.php"true301);

  • #5
    New Coder
    Join Date
    Nov 2011
    Location
    Ratio, Logic
    Posts
    59
    Thanks
    3
    Thanked 6 Times in 6 Posts
    If not disabled from indexing like Arcticwarrio proposed, bots (i.e. google bot) crawls all visible content of the page and definitely indexes new link as search.


  •  

    Posting Permissions

    • You may not post new threads
    • You may not post replies
    • You may not post attachments
    • You may not edit your posts
    •