Hello and welcome to our community! Is this your first visit?
Register
Enjoy an ad free experience by logging in. Not a member yet? Register.
Results 1 to 11 of 11
  1. #1
    Senior Coder
    Join Date
    May 2006
    Posts
    1,673
    Thanks
    28
    Thanked 4 Times in 4 Posts

    How do I download a file and unzip it with curl ?

    Hello,

    What I want to do is download file like this:
    www.support-focus.com/test_map.zip

    This is my sitemap zipped up and I want to use this as a test.

    I am going to run the php script on a different server.

    test_map is an xml file so what I want to do is download
    the test_map.zip, unzip it and rename it to sitemap.xml

    When I googled this, I found reference to a Pclzip class
    But I was thinking there might be something in the curl
    library that does it.

    Since I want to grab the file with curl, does anyone know if curl can
    unzip the file ?

    If so how do I do it with the above file.

    many thanks

  • #2
    Rockstar Coder
    Join Date
    Jun 2002
    Location
    USA
    Posts
    9,074
    Thanks
    1
    Thanked 328 Times in 324 Posts
    CURL cannot unzip files, you would need to use a different library for that. You can still use CURL to download the file first though.
    OracleGuy

  • #3
    Senior Coder
    Join Date
    May 2006
    Posts
    1,673
    Thanks
    28
    Thanked 4 Times in 4 Posts
    Thanks, I also just noticed this in the php manual:

    About using the Zip functions:

    PHP 5.2.0 and later Linux systems

    In order to use these functions you must compile PHP with zip support by using the --enable-zip configure option.

    When using the pecl install it should automatically download the latest version and install it right? So to use the pecl command do I just login with putty and at the command line type:

    $ pecl install zip

    No yum install ? Is that dollar sign supposed to be there ?

    Would much appreciate the clarification

  • #4
    Super Moderator
    Join Date
    Feb 2009
    Location
    England
    Posts
    539
    Thanks
    8
    Thanked 63 Times in 54 Posts
    If it's your own server, or you have access to it, you can always:

    PHP Code:
    exec('unzip '.$myzipfile.' /home/mysite/temp'); 
    lamped.co.uk :: Design, Development & Hosting
    marcgray.co.uk :: Technical blog

  • #5
    Master Coder
    Join Date
    Dec 2007
    Posts
    6,682
    Thanks
    436
    Thanked 890 Times in 879 Posts
    Quote Originally Posted by jeddi View Post
    Thanks, I also just noticed this in the php manual:

    About using the Zip functions:

    PHP 5.2.0 and later Linux systems

    In order to use these functions you must compile PHP with zip support by using the --enable-zip configure option.

    When using the pecl install it should automatically download the latest version and install it right? So to use the pecl command do I just login with putty and at the command line type:

    $ pecl install zip

    No yum install ? Is that dollar sign supposed to be there ?

    Would much appreciate the clarification
    in manual say how you can do in general, for a given linux distro you must use the repository of the package manager you use. Find what rpm you need and then use yum to install it.

    best regards

  • #6
    Senior Coder
    Join Date
    May 2006
    Posts
    1,673
    Thanks
    28
    Thanked 4 Times in 4 Posts
    OK, after a lot of updating of PEAR and stuff I now
    have the ZipArchive working ... a bit

    This is my code:

    PHP Code:
    <?php 

    if (!extension_loaded('zip')) {
        
    dl('zip.so');
    }

    $func_path =  "/home/guru54gt5/public_html/im/my_functions.php";
    require_once(
    "$func_path");

    $target_url "http://www.support-focus.com/test_map.zip";
    $userAgent 'Googlebot/2.1 (http://www.googlebot.com/bot.html)';

    echo 
    "<br>Starting<br>Target_url: $target_url<br><br>";

    // make the cURL request to $target_url
    $ch curl_init();
    curl_setopt($chCURLOPT_USERAGENT$userAgent);
    curl_setopt($chCURLOPT_URL,$target_url);
    curl_setopt($chCURLOPT_FAILONERRORtrue);
    curl_setopt($chCURLOPT_FOLLOWLOCATIONtrue);
    curl_setopt($chCURLOPT_AUTOREFERERtrue);
    curl_setopt($chCURLOPT_BINARYTRANSFER,true);
    curl_setopt($chCURLOPT_TIMEOUT10);
    $page curl_exec($ch);
    if (!
    $page) {
        echo 
    "<br />cURL error number:" .curl_errno($ch);
        echo 
    "<br />cURL error:" curl_error($ch);
        exit;
    }
    curl_close($ch);

    // Un zip the file 

        
    $zip = new ZipArchive;
        if (! 
    $zip) {
            echo 
    "Could not make ZipArchive object.";
            exit;
        }
        
    $zip->open("$page");
        
    $zip->extractTo('./');
        
    $zip->close();
            echo 
    "Ok!";
    ?>

    The errors I get are in fact warnings:

    Warning: ZipArchive::extractTo() [function.ZipArchive-extractTo]: Invalid or unitialized Zip object in /home/guru54gt5/public_html/sys/convert_xml_no1.php on line 40

    Warning: ZipArchive::close() [function.ZipArchive-close]: Invalid or unitialized Zip object in /home/guru54gt5/public_html/sys/convert_xml_no1.php on line 41
    Ok!

    Line 40 and 41 are:

    PHP Code:
    $zip->extractTo('./');
    $zip->close(); 
    Is the problem because my $page is not a file ?

    I am a bit confused about this so would appreciate some help.

    Thanks.

  • #7
    Senior Coder
    Join Date
    May 2006
    Posts
    1,673
    Thanks
    28
    Thanked 4 Times in 4 Posts
    I have moved forward with my script
    but I am still getting errors.

    This is what I have:
    PHP Code:
    <?php 

    if (!extension_loaded('zip')) {
        
    dl('zip.so');
    }

    $func_path =  "/home/guru54gt5/public_html/im/my_functions.php";
    require_once(
    "$func_path");

    $target_url "http://www.support-focus.com/test_map.zip";
    $userAgent 'Googlebot/2.1 (http://www.googlebot.com/bot.html)';
    $file_zip "new.zip";
    $file_txt "new.txt";

    echo 
    "<br>Starting<br>Target_url: $target_url<br><br>";


    // make the cURL request to $target_url
    $ch curl_init();
    $fp fopen("$file_zip""w"); 
    curl_setopt($chCURLOPT_USERAGENT$userAgent);
    curl_setopt($chCURLOPT_URL,$target_url);
    curl_setopt($chCURLOPT_FAILONERRORtrue);
    curl_setopt($chCURLOPT_FOLLOWLOCATIONtrue);
    curl_setopt($chCURLOPT_AUTOREFERERtrue);
    curl_setopt($chCURLOPT_BINARYTRANSFER,true);
    curl_setopt($chCURLOPT_TIMEOUT10);
    curl_setopt($chCURLOPT_FILE$fp);

    $page curl_exec($ch);


    if (!
    $page) {
        echo 
    "<br />cURL error number:" .curl_errno($ch);
        echo 
    "<br />cURL error:" curl_error($ch);
        exit;
    }
    curl_close($ch);

    echo 
    "<br>Downloaded file: $target_url";
    echo 
    "<br>Saved as file: $file_zip";
    echo 
    "<br>About to unzip ...";

    // Un zip the file 

    $zip = new ZipArchive;
        if (! 
    $zip) {
            echo 
    "Could not make ZipArchive object.";
            exit;
        }
        
    $zip->open("$file_zip");
        
    $zip->extractTo("$file_txt");
        
    $zip->close();

    echo 
    "<br>Unzipped file to: $file_txt<br><br>";        
                
    ?>
    The result:

    Starting
    Target_url: http://www.support-focus.com/test_map.zip


    Downloaded file: http://www.support-focus.com/test_map.zip
    Saved as file: new.zip
    About to unzip ...
    Warning: ZipArchive::extractTo() [function.ZipArchive-extractTo]: Invalid or unitialized Zip object in /home/guru54gt5/public_html/sys/convert_xml_no2.php on line 52

    Warning: ZipArchive::close() [function.ZipArchive-close]: Invalid or unitialized Zip object in /home/guru54gt5/public_html/sys/convert_xml_no2.php on line 53

    Unzipped file to: new.txt
    Even though it says that it has unzipped the file,
    it doesn't.

    Anyone know what I am doing wrong ?

  • #8
    Senior Coder djm0219's Avatar
    Join Date
    Aug 2003
    Location
    Wake Forest, North Carolina
    Posts
    1,301
    Thanks
    4
    Thanked 203 Times in 200 Posts
    Your call to cURL does not appear to be suppressing headers so what you're getting back is likely the headers of the page as well as the ZIP file itself. Adding the headers is likely the reason why your ZIP file ends up being invalid. Try adding:

    PHP Code:
    curl_setopt ($chCURLOPT_HEADER,0); 
    to your cURL options.

    You could probably see this by looking at the ZIP file that gets downloaded with a text editor. It will likely be the headers followed by the binary data that is the actual ZIP file.
    Dave .... HostMonster for all of your hosting needs

  • #9
    Senior Coder
    Join Date
    May 2006
    Posts
    1,673
    Thanks
    28
    Thanked 4 Times in 4 Posts
    Thanks for advice

    I have incorporated 2 changes:

    1)The headers have been set to 0 so as not
    to put them in the file

    2) I have checked to see if open() is working
    ( It isn't )

    Here is my script:


    PHP Code:
    if (!extension_loaded('zip')) {
        
    dl('zip.so');
    }

    $func_path =  "/home/guru54gt5/public_html/im/my_functions.php";
    require_once(
    "$func_path");

    $target_url "http://www.support-focus.com/test_map.zip";
    $userAgent 'Googlebot/2.1 (http://www.googlebot.com/bot.html)';
    $file_zip "new.zip";
    $file_txt "new.txt";

    echo 
    "<br>Starting<br>Target_url: $target_url";
    echo 
    "<br>Headers stripped out";

    // make the cURL request to $target_url
    $ch curl_init();
    $fp fopen("$file_zip""w"); 
    curl_setopt($chCURLOPT_USERAGENT$userAgent);
    curl_setopt($chCURLOPT_URL,$target_url);
    curl_setopt($chCURLOPT_FAILONERRORtrue);
    curl_setopt($chCURLOPT_HEADER,0); 
    curl_setopt($chCURLOPT_FOLLOWLOCATIONtrue);
    curl_setopt($chCURLOPT_AUTOREFERERtrue);
    curl_setopt($chCURLOPT_BINARYTRANSFER,true);
    curl_setopt($chCURLOPT_TIMEOUT10);
    curl_setopt($chCURLOPT_FILE$fp);

    $page curl_exec($ch);


    if (!
    $page) {
        echo 
    "<br />cURL error number:" .curl_errno($ch);
        echo 
    "<br />cURL error:" curl_error($ch);
        exit;
    }
    curl_close($ch);

    echo 
    "<br>Downloaded file: $target_url";
    echo 
    "<br>Saved as file: $file_zip";
    echo 
    "<br>About to unzip ...";

    // Un zip the file 

    $zip = new ZipArchive;
        if (! 
    $zip) {
            echo 
    "<br>Could not make ZipArchive object.";
            exit;
        }
        if(
    $zip->open("$file_zip") != "true") {
                echo 
    "<br>Could not open $file_zip";
                    }
        
    $zip->extractTo("$file_txt");
        
    $zip->close();

    echo 
    "<br>Unzipped file to: $file_txt<br><br>";        
                
    ?> 

    And the result:

    Starting
    Target_url: http://www.support-focus.com/test_map.zip
    Headers stripped out
    Downloaded file: http://www.support-focus.com/test_map.zip
    Saved as file: new.zip
    About to unzip ...
    Could not open new.zip
    Warning: ZipArchive::extractTo() [function.ZipArchive-extractTo]: Invalid or unitialized Zip object in /home/guru54gt5/public_html/sys/convert_xml_no2.php on line 55

    Warning: ZipArchive::close() [function.ZipArchive-close]: Invalid or unitialized Zip object in /home/guru54gt5/public_html/sys/convert_xml_no2.php on line 56

    Unzipped file to: new.txt
    So the ope() is not working - but I do not know why

    The file is only a small sitemap file zipped up.

    So I opened the downloaded file new.zip with notepad

    This is what it contains:

    PK  ZO;hmM 2 test_map.xmlWr0}+\^;MnM az{ʨ5$c IMi+X2:g/^W번Vh"}nZRn>U֦:Q{ܹjMrX,c+sx0Nz}ڪ(Iׯf*?(m} *l~ER(N^O{X6 "iP8̢F<VJc4FSh#y~kxTF i4eVÂdO*2$<ě\%. ~M30{
    *(I90~vq JgUy'I0 ]%@0|[T/I;!]0JSW+
    a]IY] Oy |=gMVVBn0kgBM0.8% W :( JI-\# c!4hNjC9Dp8]m4B%P sâTu]j,O^Hz]"|.`Lz7U#< ^ktOG3}"t8j0&TI#myE8wy^-LvL漱| أ^]<賷dQj薓%5>Y-"662*{xL!&Ӽs!LPK   ZO;hmM 2  test_map.xmlPK   : 
    Any ideas why things are not working ?

  • #10
    Senior Coder djm0219's Avatar
    Join Date
    Aug 2003
    Location
    Wake Forest, North Carolina
    Posts
    1,301
    Thanks
    4
    Thanked 203 Times in 200 Posts
    Are you able to open the downloaded ZIP file without errors (not from your program)?
    Dave .... HostMonster for all of your hosting needs

  • #11
    Senior Coder
    Join Date
    May 2006
    Posts
    1,673
    Thanks
    28
    Thanked 4 Times in 4 Posts
    Good question.

    The answer is YES

    I ftpped the downloaded zip file over to my pc
    and then unzipped it.

    No problem with unzipping the file


  •  

    Posting Permissions

    • You may not post new threads
    • You may not post replies
    • You may not post attachments
    • You may not edit your posts
    •