...

View Full Version : Stop Multiple Entries on a Flat File



byrondallas
07-28-2012, 01:14 PM
Hey Guys, I subscribe to a free cron service that hits my server load page once an hour, 24 hours a day, but sometimes the bot hits it twice with just a few seconds in between. Every time it hits my page I grab the server load, date and time and write it to a flat file. For the most part it only pings it once an hour but sometimes I'll get two or three entries right on top of each other. At the end of the last month it was really bad:

This is normal:


Sat 30th Jun, 12:29:26 am | Server Load:*5.76
Sat 30th Jun, 01:29:25 am | Server Load:*4.64
Sat 30th Jun, 02:29:25 am | Server Load:*3.51
Sat 30th Jun, 03:29:25 am | Server Load:*3.91
Sat 30th Jun, 04:29:24 am | Server Load:*4.31
Sat 30th Jun, 05:29:25 am | Server Load:*5.24
Sat 30th Jun, 06:29:25 am | Server Load:*4.50
Sat 30th Jun, 07:29:24 am | Server Load:*4.85
Sat 30th Jun, 08:29:29 am | Server Load:*5.59
Sat 30th Jun, 09:29:26 am | Server Load:*4.80
Sat 30th Jun, 10:29:24 am | Server Load:*6.23
Sat 30th Jun, 11:29:26 am | Server Load:*6.06
Sat 30th Jun, 12:29:26 pm | Server Load:*4.60
Sat 30th Jun, 01:29:25 pm | Server Load:*4.45
Sat 30th Jun, 02:29:25 pm | Server Load:*3.83
Sat 30th Jun, 03:29:25 pm | Server Load:*4.18
Sat 30th Jun, 04:29:24 pm | Server Load:*3.36
Sat 30th Jun, 05:29:58 pm | Server Load:*4.11


Then it went crazy:


Sat 30th Jun, 05:30:55 pm | Server Load:*3.98
Sat 30th Jun, 05:31:52 pm | Server Load:*3.94
Sat 30th Jun, 05:32:49 pm | Server Load:*3.85
Sat 30th Jun, 05:34:52 pm | Server Load:*3.74
Sat 30th Jun, 05:37:11 pm | Server Load:*3.97
Sat 30th Jun, 05:38:07 pm | Server Load:*3.96
Sat 30th Jun, 05:39:08 pm | Server Load:*3.83
Sat 30th Jun, 05:39:51 pm | Server Load:*3.76
Sat 30th Jun, 05:40:56 pm | Server Load:*3.67
Sat 30th Jun, 05:41:54 pm | Server Load:*3.90
Sat 30th Jun, 05:43:06 pm | Server Load:*3.76
Sat 30th Jun, 05:45:09 pm | Server Load:*3.54
Sat 30th Jun, 05:47:13 pm | Server Load:*3.65
Sat 30th Jun, 05:48:05 pm | Server Load:*3.60
Sat 30th Jun, 05:49:18 pm | Server Load:*3.49
Sat 30th Jun, 05:50:12 pm | Server Load:*3.43
Sat 30th Jun, 05:51:09 pm | Server Load:*3.33
Sat 30th Jun, 05:51:58 pm | Server Load:*3.52
Sat 30th Jun, 05:53:02 pm | Server Load:*3.49
Sat 30th Jun, 05:53:55 pm | Server Load:*3.53
Sat 30th Jun, 05:54:52 pm | Server Load:*3.47
Sat 30th Jun, 05:56:10 pm | Server Load:*3.32
Sat 30th Jun, 05:57:00 pm | Server Load:*3.38
Sat 30th Jun, 05:58:07 pm | Server Load:*3.34
Sat 30th Jun, 05:59:09 pm | Server Load:*3.44
Sat 30th Jun, 05:59:33 pm | Server Load:*3.38
Sat 30th Jun, 06:00:53 pm | Server Load:*3.27
Sat 30th Jun, 06:01:43 pm | Server Load:*3.42
Sat 30th Jun, 06:02:40 pm | Server Load:*3.33
Sat 30th Jun, 06:03:37 pm | Server Load:*3.28
Sat 30th Jun, 06:04:39 pm | Server Load:*3.20
Sat 30th Jun, 06:05:38 pm | Server Load:*3.20
Sat 30th Jun, 06:06:34 pm | Server Load:*3.18
Sat 30th Jun, 06:07:37 pm | Server Load:*3.14
Sat 30th Jun, 06:08:44 pm | Server Load:*3.06
Sat 30th Jun, 06:29:44 pm | Server Load:*3.30
Sat 30th Jun, 07:29:29 pm | Server Load:*2.66
Sat 30th Jun, 08:29:31 pm | Server Load:*2.58
Sat 30th Jun, 09:29:34 pm | Server Load:*3.32
Sat 30th Jun, 10:29:33 pm | Server Load:*6.43
Sat 30th Jun, 11:29:31 pm | Server Load:*3.59


What can I do once the cron service hits my page to keep it from making a double entry (or multi entry) until the next hour?

Fou-Lu
07-28-2012, 02:41 PM
PHP wise whatever script you are using to log these entries can open the file first, read the last entry and if its been within the past 50 minutes or so then skip the log. That of course is only useful if its PHP doing the work. Make sure you record elsewhere that its been skipped though, otherwise you won't know there is a problem.

Why not just use your own cron service?

byrondallas
07-28-2012, 03:55 PM
Here's what I'm using to detect the cron bot.


######### If SetCronJob add stats #########
$ip = $_SERVER['REMOTE_ADDR'];
$agent = $_SERVER['HTTP_USER_AGENT'];
$host = gethostbyaddr($ip);
if ($agent == "SetCronJob/1.0 (+http://www.setcronjob.com/)")
{

######### write stats to data file #########
$month = date("F_Y");
$data_file = "server_data/server_data_$month.dat";

$time = date("D dS M, h:i:s a");
$today = date("D");
$stats = "<li class=\"$today\">$time | Server Load:&#160;$loadavg</li>\n";
file_put_contents($data_file, $stats, FILE_APPEND);
######### end stats #########
}




read the last entry and if its been within the past 50 minutes or so then skip the log.

What would be the best way to do this?


Why not just use your own cron service?

I'm hosting with a free web host and they only allow two cron jobs per day.

byrondallas
07-28-2012, 09:13 PM
Thanks for the time idea. This seems to work perfectly:


if ( is_file($data_file) )
{
$age = time()-filemtime($data_file);

$min = 5;
$expire = 60*$min;

if ( $age > $expire )
{
file_put_contents($data_file, $stats, FILE_APPEND);
}
}

Fou-Lu
07-29-2012, 03:28 PM
Ah yeah I didn't even think about that.
Pulling the modified time is much easier and faster than reading the file for the last entry.



EZ Archive Ads Plugin for vBulletin Copyright 2006 Computer Help Forum