...

View Full Version : Asynchronous HTTP Requests



Will Bontrager
08-19-2012, 03:03 PM
Is there a way to create multiple HTTP connections with PHP and handle the request responses in whatever order they are received?

Sure would appreciate pointers or links to tutorials.

Thanks!

Will

byrondallas
08-19-2012, 07:05 PM
You should be able to do what you want with curl, with something like this. This is only set to execute two urls but can be modified to what you want.


<?php

// create both cURL resources
$ch1 = curl_init();
$ch2 = curl_init();

// set URL and other appropriate options
curl_setopt($ch1, CURLOPT_URL, "http://url_one.html");
curl_setopt($ch1, CURLOPT_HEADER, 0);

curl_setopt($ch2, CURLOPT_URL, "http://url_two.html");
curl_setopt($ch2, CURLOPT_HEADER, 0);

//create the multiple cURL handle
$mh = curl_multi_init();

//add the two handles
curl_multi_add_handle($mh,$ch1);
curl_multi_add_handle($mh,$ch2);
$active = null;

//execute the handles
do {
$mrc = curl_multi_exec($mh, $active);
} while ($mrc == CURLM_CALL_MULTI_PERFORM);
while ($active && $mrc == CURLM_OK) {
if (curl_multi_select($mh) != -1) {
do {
$mrc = curl_multi_exec($mh, $active);
} while ($mrc == CURLM_CALL_MULTI_PERFORM);
}
}

//close the handles
curl_multi_remove_handle($mh, $ch1);
curl_multi_remove_handle($mh, $ch2);
curl_multi_close($mh);
?>

Will Bontrager
08-21-2012, 12:50 AM
Exactly what I was looking for, a place to start.

Thank you very much, byrondallas.

Will

byrondallas
08-21-2012, 03:46 AM
Exactly what kind of responses were you hoping to to get from the urls? Maybe I or somebody else can show you how using curl if your not already familiar with curl.

tangoforce
08-21-2012, 09:31 AM
Just to add, using multiple cURL connections is actually a very good way of getting around the multiple thread / fork issue on both windows and linux. I've had to do this myself and it works a treat. It will use more resources but it works very well if you're calling scripts on the localhost multiple times to execute tasks at the same time. Minimal fuss with threads, and minimal fuss with forking. You may however want to password protect those 'task scripts' and post the password to them so that external users can't abuse them etc.

Will Bontrager
08-21-2012, 01:30 PM
Exactly what kind of responses were you hoping to to get from the urls? Maybe I or somebody else can show you how using curl if your not already familiar with curl.

Right, I have never used curl before. Knowing which functions to focus on first during my research would save me a lot of time.

What I'm doing is grabbing a bunch of RSS feeds (all http://... URLs). Request one, wait for response, process. Request next one, wait for response, process. etc.

My idea when I started this thread was to request additional RSS feeds while waiting for response from other requests.

Currently, it's taking a minute or more to grab and process the list. The list of feeds will only get larger. I considered grabbing the feeds in chunks of 10 and then relaunch the script for the next 10 but, well, it's just not an elegant solution.

I appreciate your time.

Will

tangoforce
08-21-2012, 02:09 PM
a
Right, I have never used curl before. Knowing which functions to focus on first during my research would save me a lot of time.

Until my first time I had also never used cURL before - somehow I managed ;):D:thumbsup:

If you want to know about the cURL functions and how to use them then you will almost certainly find this link useful: http://uk.php.net/curl

Remember that checking out the user comments on each functions page always reveals some good code that you can use too.

with regards to your multiple feeds then yes using multiple curl handles / execs would be a good way to go as the processeing can be done in parallel like a threaded system.

Will Bontrager
08-23-2012, 02:27 PM
byrondallas and tangoforce and anyone else who may be following this thread, this seems to do a good job. Although I don't have a lot to compare it to.

http://code.google.com/p/rolling-curl/source/browse/#svn/trunk

It works. And it is well commented for those who, like me, enjoy learning how things are done.

Will



EZ Archive Ads Plugin for vBulletin Copyright 2006 Computer Help Forum