View Full Version : Staggering XML data requests.

05-18-2011, 05:30 PM
Currently I am farming sites like http://us.battle.net/wow/en/character/$realm/$char/simple for information and then adding that information to a MySQL db every time a character is added. However the company that owns battle.net (Blizzard Activision) sent me a polite e-mail detailing that I can only make 2000 requests per day and cannot make 2 requests within 3 seconds of each other. Does anyone have any ideas on how to accomplish this?
Current version : http://pastebin.com/4zWrytAd lines 79-211
My thoughts :

I could put all the requests into a database, and run a php chon job every 15 minutes doing at most 20 requests. Run each of them with 43 seconds of sleep() would yield 1920 a day and no requests within 3 seconds of each other. And that would probably work. But it's.... ghetto? Poorly executed might be a better word. I'm looking for something more elegant.


05-20-2011, 05:34 AM
Bump for the need of elegance.

05-21-2011, 07:24 AM
Bump in compliance with rules.