...

View Full Version : Curl - saving complete webpage into a variable



kevinkhan
03-01-2012, 01:43 PM
I have this code



curl_setopt($ch, CURLOPT_POST, 0);
curl_setopt($ch, CURLOPT_URL, $link);
$wall = curl_exec($ch);


but the $link is a facebook wall page that takes a couple of seconds to load fully. and you have to scroll down the page before more loads or you have to click older posts

when i run the script the $wall doesnt have the full page.

Is there a way that i can delay the $link from saving the file so that it reads more of the page??

Fou-Lu
03-01-2012, 02:42 PM
Curl shouldn't be resuming its control to the script until either its timed out or its complete. You may want to check first if it returns a 200, if it does then that sounds like its working fine. Otherwise, you can set a new option for CURLOPT_TIMEOUT in the number of seconds.
Are you sure its not javascript that's populating the wall? Sounds to me like you are trying to wait for the javascript.

Keleth
03-01-2012, 03:15 PM
Yup, Facebook populates and extends its wall with javascript, so this method won't work.

If you want to grab a wall, the Facebook API does that just fine.

kevinkhan
03-05-2012, 05:37 PM
Curl shouldn't be resuming its control to the script until either its timed out or its complete. You may want to check first if it returns a 200, if it does then that sounds like its working fine. Otherwise, you can set a new option for CURLOPT_TIMEOUT in the number of seconds.
Are you sure its not javascript that's populating the wall? Sounds to me like you are trying to wait for the javascript.

yes its waiting for js to load. but i think the curl is saving the page to quickley :(

does the facebook api allow you to save content from a wall of none friends?

and can somebody point in to a link where i can read about the api.

Fou-Lu
03-05-2012, 05:57 PM
yes its waiting for js to load. but i think the curl is saving the page to quickley :(

does the facebook api allow you to save content from a wall of none friends?

and can somebody point in to a link where i can read about the api.

Can't help you with the facebook api, but I'm sure they would accommodate this. Unless you write an ECMA parser yourself, JS will not execute when retrieved by curl.

Keleth
03-05-2012, 06:09 PM
For the facebook API, just google "Facebook API". Plenty of pages.

And no, the API only allows you to pull wall posts that are public or give you permission (as a friend or as an app).



EZ Archive Ads Plugin for vBulletin Copyright 2006 Computer Help Forum