View Full Version : Identifying Google Bots and others

07-25-2008, 07:00 PM
Hey guys,

I have an online list that tells me how many users are online. It also display which of my registered users are online too.

What i'd like to do is get my script to recognised bots like google aswell and add them to my list.

My code is something like this:

$timestamp = time();
$timeout = $timestamp - 900;

$d=mysql_query("UPDATE `Member` SET `timest` = '$timestamp' WHERE `ID` ='$userid2'") or die(mysql_error());

//Insert User
$insert = mysql_query("INSERT INTO online (timestamp, ip, file) VALUES('$timestamp','".$_SERVER['REMOTE_ADDR']."','".$_SERVER['PHP_SELF']."')") or die("Error in who's online insert query!");

$result = mysql_query("SELECT DISTINCT ip FROM online") or die("Error in who's online result query!");
$users = mysql_num_rows($result);
//Display all users who have been online within the last 5minutes.
$sel=mysql_query("SELECT * FROM `Member` WHERE `timest` > '$timeout' ORDER BY `ID`") or die(mysql_error());
$users = $users - $n;
echo"Members Online: $n | Guests: $users<br> ";

How would i try and recognise google bots? Does google use a specific I.P. that i have to register or something :X?

Thanks for your help.

07-26-2008, 11:35 PM
try an allready built program for this
I second AW stats. It basically tells you what robots visited you, when the last one visited (time/date), how much bandwidth they consumed for the month, etc. Very handy if you dont' want to dig through the log files yourself.

If you have access to the log files, there's a program called Windows Grep which is basically a text analyzer, but it will let you search huge log files for text strings like "googlebot" or whatever you want, and it will only return those lines with "googlebot" in it, so you can see every single page that was requested, when, by what IP, etc.

obviusly if you find how they work you could take this on for urself.

07-27-2008, 12:33 AM
use crawltrack (http://www.crawltrack.fr/) it's really good at tracking a huge list of robots