Hello and welcome to our community! Is this your first visit?
Register
Enjoy an ad free experience by logging in. Not a member yet? Register.
Results 1 to 12 of 12
  1. #1
    Senior Coder
    Join Date
    May 2006
    Posts
    1,674
    Thanks
    28
    Thanked 4 Times in 4 Posts

    Why use curl rather than fopen() and fread() ?

    Sorry - I am a bit of a newbie with this curl stuff.

    Funny thing is, I have been using curl but now I come to
    want to code up something from scratch I have to ask:
    why is curl better than using fopen() and fread() ?

    I have got the Snoopy class as well if I want to use it
    and I just got the "elHttpClient" with is a cUrl addon
    class to supposedly make things easier.

    Now I don't know what to use and why

    Can someone please help explaing what the differences are ?

    Thanks.

  • #2
    Mega-ultimate member
    Join Date
    Jun 2002
    Location
    Winona, MN - The land of 10,000 lakes
    Posts
    1,855
    Thanks
    1
    Thanked 45 Times in 42 Posts
    I may not be 100% right here, but in general fread, fopen are going to send HTTP GET requests, where as curl can be used to send either HTTP GET or HTTP POST requests.

  • #3
    Senior Coder
    Join Date
    May 2006
    Posts
    1,674
    Thanks
    28
    Thanked 4 Times in 4 Posts
    Thanks for your answer.

    I also now that with cUrl you can select your agent
    and referer, but I don't really see how important that is
    - does that mean that you can "cloak" your identity from the
    server you visit ?

  • #4
    Senior Coder
    Join Date
    May 2006
    Posts
    1,674
    Thanks
    28
    Thanked 4 Times in 4 Posts
    If anyone else can offer some insight
    it would be appreciated.

    Maybe the main difference is being able to submit forms using cUrl ?

    (But then Snoopy does that without cUrl, doesn't it ? )

    Please help as I really want to grasp this difference and use the best method.

    thanks

  • #5
    Senior Coder kbluhm's Avatar
    Join Date
    Apr 2007
    Location
    Philadelphia, PA, USA
    Posts
    1,509
    Thanks
    3
    Thanked 258 Times in 254 Posts
    It's also much faster, and does not rely on allow_url_fopen.

  • #6
    Senior Coder
    Join Date
    May 2006
    Posts
    1,674
    Thanks
    28
    Thanked 4 Times in 4 Posts
    Well that is an important point.

    So how much faster is cUrl over fopen() and fread() ?

    Are we talking about marginally faster ( like 10 % )
    or factorially faster ( like 3 x faster ) ?

    I want to chose the best one for crawling the web and developing my own index. I will open a thread on that subject today.

    Any one know more about this ?

  • #7
    Senior Coder kbluhm's Avatar
    Join Date
    Apr 2007
    Location
    Philadelphia, PA, USA
    Posts
    1,509
    Thanks
    3
    Thanked 258 Times in 254 Posts
    Well why not research this yourself?

    Create a file for each option -- curl, fopen, file_get_contents, or fsockopen -- and just benchmark dead requests within a for-loop, using microtime() to benchmark the execution time.
    Last edited by kbluhm; 10-22-2009 at 01:59 PM.

  • #8
    God Emperor Fou-Lu's Avatar
    Join Date
    Sep 2002
    Location
    Saskatoon, Saskatchewan
    Posts
    16,987
    Thanks
    4
    Thanked 2,660 Times in 2,629 Posts
    Quote Originally Posted by kbluhm View Post
    Well why not research this yourself?

    Create a file for each option -- curl, fopen, file_get_contents, or fsockopen -- and just benchmark dead requests within a for-loop, using microtime() to benchmark the execution time.
    And post you're results too, I'm a little curious. Given that these are all stream handlers, I'm a little curious what beats what (my suspicion: fsockopen > curl > fopen >>>>>> file_get_contents [since this one I expect to be the slowest])
    PHP Code:
    header('HTTP/1.1 420 Enhance Your Calm'); 

  • #9
    Regular Coder FWDrew's Avatar
    Join Date
    Apr 2008
    Location
    Missouri
    Posts
    380
    Thanks
    38
    Thanked 45 Times in 43 Posts
    Quote Originally Posted by Fou-Lu View Post
    And post you're results too, I'm a little curious. Given that these are all stream handlers, I'm a little curious what beats what (my suspicion: fsockopen > curl > fopen >>>>>> file_get_contents [since this one I expect to be the slowest])
    Here is a little data for you all, from what I have seen, cURL is much faster:

    http://stackoverflow.com/questions/5...er-performance
    Dev-Tips.com || Employee at the Envato Network || My Portfolio

  • #10
    Senior Coder
    Join Date
    May 2006
    Posts
    1,674
    Thanks
    28
    Thanked 4 Times in 4 Posts
    That was very useful.

    I read a lot of the surrounding links and learnt a lot.

  • #11
    New to the CF scene
    Join Date
    Dec 2009
    Posts
    1
    Thanks
    0
    Thanked 0 Times in 0 Posts
    About performance, I've just read this from http://forums.winamp.com/showthread.php?postid=2338006 :
    http://ca3.php.net/manual/en/ref.curl.php#75126
    "I have created a script that tests all 4 methods on 20 different websites 10 times each. The results are clear, here is one set of the results:

    Curl time: 43.02 seconds
    FGC time: 86.48 seconds
    Fopen time: 86.34 seconds
    Socket time: 44.91 seconds

    As you can see, there is little performance difference between file_get_contents and fopen/fread. Likewise, there is little difference between curl and sockets."


    Anyway, apart from the speed difference and the ability for cURL to make GET and POST requests, are there other differences?
    (And in addition, what would be the IP passed, the server one?)

    (Please excuse me if I made some mistakes, I'm not english ^^)
    Last edited by ricorine; 12-18-2009 at 09:23 PM.

  • #12
    Super Moderator
    Join Date
    Feb 2009
    Location
    England
    Posts
    539
    Thanks
    8
    Thanked 63 Times in 54 Posts
    It doesn't matter which is faster if you want to write your own crawler. You should specify your own User-Agent and manage your own Referrer fields, which requires either cURL or your own totally custom socket handling functions.
    lamped.co.uk :: Design, Development & Hosting
    marcgray.co.uk :: Technical blog


  •  

    Posting Permissions

    • You may not post new threads
    • You may not post replies
    • You may not post attachments
    • You may not edit your posts
    •