Hello and welcome to our community! Is this your first visit?
Register
Enjoy an ad free experience by logging in. Not a member yet? Register.
Results 1 to 4 of 4

Thread: CURL Help

  1. #1
    New Coder
    Join Date
    Nov 2007
    Location
    St.Louis MO
    Posts
    15
    Thanks
    3
    Thanked 0 Times in 0 Posts

    CURL Help

    Hey guys,

    OK, here's what I am trying to do: I need to log into another website, and download their reports. Seems easy right? Well, it is... because I've done it with numerous other websites.

    However, this one is tricky. The only way I can download the report at this website is by having the correct "key" at the end of the URL. The key is generated by their server and carries over with each link that you click on their site once logged in. Sort of like a session ID, but it's not a session ID at all.

    Anyway, this key is unique to the current cookie session for that user. If they were to log out, none of those links with that unique trailing "key" would work. They would have to log back in and use the new links. The trailing key is nothing but 10 numbers long.

    So, I've devised a small curl script that does everything... it logs into the site, grabs that trailing "key" from a hidden text field, and it downloads the CSV report. However, when I look at the downloaded report, it shows a "This page does not exist" custom apache page from their server.

    The problem is that it is either losing the cookie when it goes to download the report URL, or it logs out/back in when it goes to download the report URL.

    Here's the code (I've removed the URL's):
    PHP Code:
    $graburl = **URL OF CSV REPORT**;
    $cookiefile tempnam("/tmp""cookies"); 
    $ch curl_init();
    curl_setopt($chCURLOPT_URL$loginurl);
    curl_setopt($chCURLOPT_COOKIEFILE$cookiefile);
    curl_setopt($chCURLOPT_COOKIEJAR$cookiefile);
    curl_setopt($chCURLOPT_POST1);
    curl_setopt($chCURLOPT_POSTFIELDS"$usernamefield=$username&$passwordfield=$password");

    ob_start();
    $output curl_exec($ch);
    ob_end_clean();

    ***************
    ADD CODE***************
    $link '**URL OF REPORT PAGE TO FETCH THE KEY**';
    curl_setopt($chCURLOPT_URL$link);
    curl_setopt($chCURLOPT_RETURNTRANSFER1);
    $page curl_exec($ch);

    $sk '%<input type="hidden" id="window_name" value="(.*)">%';
    preg_match_all($sk$page$resultsPREG_PATTERN_ORDER);
    ***************
    ADD CODE***************

    if (
    $output == '1') {
    curl_setopt($chCURLOPT_URL"$graburl&session_key=".$results[1][0]);
    $putcsv "report.csv";
    curl_setopt($chCURLOPT_HEADER0);
    curl_setopt($chCURLOPT_COOKIEFILE$cookiefile);
    curl_setopt($chCURLOPT_COOKIEJAR$cookiefile);
    curl_setopt($chCURLOPT_RETURNTRANSFER1);
    $result curl_exec($ch);
    file_put_contents("**MY URL**".$putcsv."""$result");
    }
    curl_close($ch); 
    Note: This code works beautifully on other sites where I get reports from as well. It's just the extra code marked in "**ADD CODE**" and the "&session_key=".$results[1][0]" added to the graburl that is giving me problems.

    It may have something to do with grabbing to URL's in the same curl session, I can't figure it out.

    Any help is appreciated,

    Thanks,
    Corey

  • #2
    Regular Coder lokeshshettyk's Avatar
    Join Date
    Aug 2008
    Location
    On the way to the moon!
    Posts
    157
    Thanks
    5
    Thanked 20 Times in 20 Posts
    I am not sure if this helps, but I always do this with the init()
    PHP Code:
    $ch curl_init ("http://somedomain.com/"); // domain from which you are trying to retrieve the file 
    _______________________________________________________________________________________
    Thank the user for his/her helpful post by clicking on the button below the post :)

    Cheers :)
    ~ LSK ~

  • #3
    New Coder
    Join Date
    Nov 2007
    Location
    St.Louis MO
    Posts
    15
    Thanks
    3
    Thanked 0 Times in 0 Posts
    Quote Originally Posted by lokeshshettyk View Post
    I am not sure if this helps, but I always do this with the init()
    PHP Code:
    $ch curl_init ("http://somedomain.com/"); // domain from which you are trying to retrieve the file 
    Yeah, I tried that also. Same results.

    Thanks for the reply.

  • #4
    New Coder
    Join Date
    Nov 2007
    Location
    St.Louis MO
    Posts
    15
    Thanks
    3
    Thanked 0 Times in 0 Posts
    Just an update:

    I've tried it with just one instance of the curl cookie options and I've tried it with init(). EX:
    PHP Code:
    $ch curl_init ("http://somedomain.com/"); 
    Neither worked.

    If any one has any suggestions or knows a better to do this, let me know.

    Thanks,
    Corey


  •  

    Posting Permissions

    • You may not post new threads
    • You may not post replies
    • You may not post attachments
    • You may not edit your posts
    •