Hello and welcome to our community! Is this your first visit?
Register
Enjoy an ad free experience by logging in. Not a member yet? Register.
Results 1 to 3 of 3
  1. #1
    Regular Coder
    Join Date
    Jan 2007
    Posts
    217
    Thanks
    9
    Thanked 0 Times in 0 Posts

    Identifying Google Bots and others

    Hey guys,

    I have an online list that tells me how many users are online. It also display which of my registered users are online too.

    What i'd like to do is get my script to recognised bots like google aswell and add them to my list.

    My code is something like this:

    PHP Code:
        $timestamp time();
    $timeout $timestamp 900;

    $d=mysql_query("UPDATE `Member` SET `timest` = '$timestamp' WHERE `ID` ='$userid2'") or die(mysql_error());

    //Insert User
    $insert mysql_query("INSERT INTO online (timestamp, ip, file) VALUES('$timestamp','".$_SERVER['REMOTE_ADDR']."','".$_SERVER['PHP_SELF']."')") or die("Error in who's online insert query!");

    $result mysql_query("SELECT DISTINCT ip FROM online") or die("Error in who's online result query!");
    $users mysql_num_rows($result);
          
    //Display all users who have been online within the last 5minutes.
          
    $sel=mysql_query("SELECT * FROM `Member` WHERE `timest` > '$timeout' ORDER BY `ID`") or die(mysql_error());
          
    $n=mysql_num_rows($sel);
          
    $users $users $n;
          echo
    "Members Online: $n | Guests: $users<br> "
    How would i try and recognise google bots? Does google use a specific I.P. that i have to register or something :X?

    Thanks for your help.

  • #2
    Regular Coder
    Join Date
    Nov 2006
    Posts
    601
    Thanks
    1
    Thanked 2 Times in 2 Posts
    try an allready built program for this
    I second AW stats. It basically tells you what robots visited you, when the last one visited (time/date), how much bandwidth they consumed for the month, etc. Very handy if you dont' want to dig through the log files yourself.

    If you have access to the log files, there's a program called Windows Grep which is basically a text analyzer, but it will let you search huge log files for text strings like "googlebot" or whatever you want, and it will only return those lines with "googlebot" in it, so you can see every single page that was requested, when, by what IP, etc.

    obviusly if you find how they work you could take this on for urself.

  • #3
    Regular Coder
    Join Date
    Oct 2003
    Posts
    603
    Thanks
    2
    Thanked 1 Time in 1 Post
    use crawltrack (http://www.crawltrack.fr/) it's really good at tracking a huge list of robots


  •  

    Posting Permissions

    • You may not post new threads
    • You may not post replies
    • You may not post attachments
    • You may not edit your posts
    •