...

View Full Version : Most reliable way of retrieving file from remote server?



spman
08-14-2012, 08:55 AM
Hi all - I am writing a PHP script that will be installed on a users server. I want to pull a file from MY remote server (a zip file) and write it to the local server that the script is running on.

I can do it just fine use file_get_contents and also using Snoopy, fsockopen or CURL.

However, the issue is that my script will be distributed to lots of people who will have a variety of servers.
They will range from freebie webspace to their own dedicated rack with everything in between :)
I can't be certain that the correct options will be enabled for file_get_contents/Snoopy/fsock/CURL to work and I know that, if certain options are disabled, it will be unlikely that the user will be able to get them enabled.

So, I am thinking that, pretty much, the only thing I can guarantee is the they will have PHP 4 + what is the best way to pull the remote file - IE: Which way has the best chance of working on such a large range of webservers :

file_get_contents
Snoopy (using fetch)
fsockopen
CURL

Any ideas or comments would be MUCH appreciated :)

IFeelYourPain
08-14-2012, 09:47 AM
Are you just looking at transferring file from server to another?
connection.php

<?php
$server = "domain.com"; //target server
$user = "ftpusername@domain.com"; //target user
$pass = "password"; //password on target server for Ftp
?>

transfer.php

<?php
require_once('connection.php');
$file = "file.zip";
$local_file = 'local.zip';
$sessid = ftp_connect($server);
$login_ok = ftp_login($sessid, $user, "$pass");
if ((!$sessid) || (!$login_ok)):
echo "failed to connect: check hostname, username & password";
exit; //failed? Unable to connect!
endif;

if (ftp_get($sessid, $local_file, $file, FTP_BINARY))
{
echo "Successfully written\n";
} else {
echo "There was a issue\n";
}

ftp_close($sessid);
?>

Fou-Lu
08-14-2012, 02:15 PM
Of the four you've posted, sockets are the most reliable. Curl is an extension and snoopy is a package. File_get_contents has only existed as of 4.3+, and will only work remotely if allow_url_fopen is enabled. Depending on what you are doing, ftp may also be an option.
There is no need to limit yourself to one. Try by route of ease: file_get_contents, then curl, then sockets (I don't know where snoopy fits in here). If one fails, try the next until all have been exhausted.

tangoforce
08-14-2012, 03:24 PM
if certain options are disabled, it will be unlikely that the user will be able to get them enabled.

So, I am thinking that, pretty much, the only thing I can guarantee is the they will have PHP 4 + what is the best way to pull the remote file

Erm, I hate to point this out but if the host has disabled remote functions, php version 4 or 5 you're still not going to be able to download a remote file.

I'll repeat that in slightly different wording:
It doesn't matter what version of php the server has installed. If the server administrator has disabled remote functions then you won't be able to download a remote file at all.

I can't get my head around why you think you'll still be able to :confused:

spman
08-14-2012, 09:57 PM
Erm, I hate to point this out but if the host has disabled remote functions, php version 4 or 5 you're still not going to be able to download a remote file.

I'll repeat that in slightly different wording:
It doesn't matter what version of php the server has installed. If the server administrator has disabled remote functions then you won't be able to download a remote file at all.

I can't get my head around why you think you'll still be able to :confused:

Hi tangoforce - thanks for the post - I realise that and, under that circumstance, I would have to resort to asking them to DL everything and upload it via FTP - I just wanted to know what the order of precedence should be for likleyhood of a particular method working.

I guess the answer is try 'em all and if they all fail then resort to FTP.


EDIT: Let me rephrase the question - Let's assume that remote functions are enabled (otherwise nothing but local FTP will work anyway) - In what order are the other methods likely to work (IE: what other dependencies in the server config could affect them) ?
I am guessing that if the server admin has disabled remote function then none of the methods will work - In your experience, how common is it to have that functionality disabled?

Fou-Lu
08-14-2012, 10:49 PM
allow_url_fopen I find is most often disabled (although somewhat rare even, its allow_url_include that should be disabled), but sockets and curl are often available. Socket's I'd deem most reliable as curl is an extension on the PHP which may or may not be compiled in.
For ease, I'd try file_get_contents if the size is small enough, then I'd try curl, then sockets. Sockets are more complicated since you have to send the instructions for the http get/post.
You can always use ini_get calls, function_exists calls, and extension_loaded calls prior to doing anything. This helps to determine what you can do. So if PHP version is > 4.3 and allow_url_fopen is enabled, you can use file_get_contents. If is < 4.3 (won't go back to 3.0 documentation; sockets I recall being there but can't verify as the API lists 4.0+, but I no longer deem 3.x as a common solution, only custom code I write for such legacy machines), then sockets are a good bet. If you are on 4.0.2+, but less than 4.3, then curl can be probed for extension_loaded, and go from there.

On the other hand, you may approach it differently. Trying sockets may be the best bet as it is likely the most available feature - its built in core, not blocked by safe mode, and not a default disabled function. But it takes the most work to do what you need to do.

So short answer is, there really isn't the best way to do this. There is just different ways to do it. I'd probably go for sockets over anything else.

tangoforce
08-14-2012, 11:19 PM
Personally, if it were me and I wanted a reliable way of doing a file download on a server that didn't have it turned off..

1) cURL - Not guaranteed to be available
2) file_get_contents() - as Fou says, small files only otherwise it may be unreliable
3) sockets - most reliable but most complex to use. You will however find samples on google.

spman
08-15-2012, 08:18 AM
Thank you so much guys for your help!

Another question if I may?

The file(s) are stored currently in an Amazon S3 Bucket - so the url to access them may be something like :

http://example.s3.amazonaws.com/files/file1.zip

I'm guessing that with S3 it should still be possible to retrieve the file using sockets and port 80?

Fou-Lu
08-15-2012, 02:30 PM
Yeah, you simply issue the port connection, and then write the HTTP GET status for it. You should then read the response back; you can use it to determine the status (200 is the only valid one in http) and process the remaining results after header.

Here's an example on the API, the first one fetches data from port 80: http://ca3.php.net/manual/en/function.fsockopen.php

spman
08-16-2012, 06:47 PM
I can't believe the help you guys have been so forthcoming with! You certainly have a lot more patience then some other forums/q&a sites! So, thank you!

If sockets are going to be the most likely to be available I guess I should try that first.

Do I have this right in the dependancies etc :

1. CURL - only works if the CURL extension is enabled but it doesn't matter if allow_url_fopen or allow_url_include are disabled?

2. file_get_contents - only works if EITHER allow_url_fopen or allow_url_include are enabled ?

3. Sockets - I have no idea what they depend on to work LOL ????

Fou-Lu
08-16-2012, 07:07 PM
Curl is independent of allow_url_fopen. The two allow directives are for filesystem wrappers using fopen and include.

File_get_contents will only work if you open them in the fashion associated with the particular ini. allow_url_fopen will allow file_get_contents to work remotely. Allow_url_include has no impact on file_get_contents as its a nonparsed read of a file and not a parsed inclusion of the file. The allow_url_include is designed to include/require remote files, but IMO carries 0 value with it since you would only get the reply after it's parsed by the webhost.

Sockets are core. They don't depends on anything beyond the socket timeout (that I'm aware of). But it is a function family, so the functions can be added to the disabled_functions directive which would forbid their usage. If these are closed, typically all of the remote calls will be.

spman
08-21-2012, 08:32 PM
Ok - I have written some test code that pulls an 18k jpg from my Amazon S3 and writes it to my local server.

Using file_get_contents it takes seconds.

Using sockets it takes about a minute.

Does anyone have any ideas please why it is so much slower with sockets?

For reference, I have used a version of the code found in example four on this page :

http://www.php-mysql-tutorial.com/wikis/php-tutorial/reading-a-remote-file-using-php.aspx

EDIT: After some googling I found a reference to a similar problem here : http://www.nutt.net/2011/12/04/fsockopen-fgets-very-slow-in-php/

However, I don't really know where I should be integrating the Connection: Close into the function referenced in the other site??

Many thanks in advance for any help :)

Fou-Lu
08-22-2012, 01:38 AM
Close would replace your current connection.

spman
08-22-2012, 09:38 AM
Close would replace your current connection.

I'm not sure what you mean i'm afraid ?

Fou-Lu
08-22-2012, 04:23 PM
If you've used that link to build your fsockopen, it passes a connection of keep-alive. Replace it with a connection close.

spman
08-22-2012, 08:09 PM
If you've used that link to build your fsockopen, it passes a connection of keep-alive. Replace it with a connection close.

Thank You Fou-Lu - It now works PERFECTLY!! :):):thumbsup:



EZ Archive Ads Plugin for vBulletin Copyright 2006 Computer Help Forum