Stream FTP download to output

Question

I am trying to stream/pipe a file to the user\'s browser through HTTP from FTP. That is, I am trying to print the contents of a file on an FTP server.

This is what

Answer 1

Try:

@readfile('ftp://username:password@host/path/file');

I find with a lot of file operations it's worthwhile letting the underlying OS functionality take care of it for you.

Answer 2

(I've never met this problem myself, so that's just a wild guess ; but, maybe... )

Maybe changing the size of the ouput buffer for the "file" you are writing to could help ?

For that, see stream_set_write_buffer.

For instance :

$fp = fopen('php://output', 'w+');
stream_set_write_buffer($fp, 0);

With this, your code should use a non-buffered stream -- this might help...

Answer 3

Sounds like you need to turn off output buffering for that page, otherwise PHP will try to fit it in all memory.

An easy way to do this is something like:

while (ob_end_clean()) {
    ; # do nothing
}

Put that ahead of your call to ->get(), and I think that will resolve your issue.

Answer 4

I know this is old, but some may still think it's useful.

I've tried your solution on a Windows environment, and it worked almost perfectly:

$conn_id = ftp_connect($host);
ftp_login($conn_id, $user, $pass) or die();

$sockets = stream_socket_pair(STREAM_PF_INET, STREAM_SOCK_STREAM,
        STREAM_IPPROTO_IP) or die();

stream_set_write_buffer($sockets[0], 0);
stream_set_timeout($sockets[1], 0);

set_time_limit(0);
$status = ftp_nb_fget($conn_id, $sockets[0], $filename, FTP_BINARY);

while ($status === FTP_MOREDATA) {
    echo stream_get_contents($sockets[1]);
    flush();
    $status = ftp_nb_continue($conn_id);
}
echo stream_get_contents($sockets[1]);
flush();

fclose($sockets[0]);
fclose($sockets[1]);

I used STREAM_PF_INET instead of STREAM_PF_UNIX because of Windows, and it worked flawlessly... until the last chunk, which was false for no apparent reason, and I couldn't understand why. So the output was missing the last part.

So I decided to use another approach:

$ctx = stream_context_create();
stream_context_set_params($ctx, array('notification' =>
        function($code, $sev, $message, $msgcode, $bytes, $length) {
    switch ($code) {
        case STREAM_NOTIFY_CONNECT:
            // Connection estabilished
            break;
        case STREAM_NOTIFY_FILE_SIZE_IS:
            // Getting file size
            break;
        case STREAM_NOTIFY_PROGRESS:
            // Some bytes were transferred
            break;
        default: break;
    }
}));
@readfile("ftp://$user:$pass@$host/$filename", false, $ctx);

This worked like a charm with PHP 5.4.5. The bad part is that you can't catch the transferred data, only the chunk size.

Answer 5

Found a solution!

Create a socket pair (anonymous pipe?). Use the non-blocking ftp_nb_fget function to write to one end of the pipe, and echo the other end of the pipe.

Tested to be fast (easily 10MB/s on a 100Mbps connection) so there's not much I/O overhead.

Be sure to clear any output buffers. Frameworks commonly buffer your output.

public function echo_contents() {
    /* FTP writes to [0].  Data passed through from [1]. */
    $sockets = stream_socket_pair(STREAM_PF_UNIX, STREAM_SOCK_STREAM, STREAM_IPPROTO_IP);

    if($sockets === FALSE) {
        throw new Exception('Unable to create socket pair');
    }

    stream_set_write_buffer($sockets[0], 0);
    stream_set_timeout($sockets[1], 0);

    try {
        // $this->ftp is an FtpConnection
        $get = $this->ftp->get_non_blocking($this->path, $sockets[0]);

        while(!$get->is_finished()) {
            $contents = stream_get_contents($sockets[1]);

            if($contents !== false) {
                echo $contents;
                flush();
            }

            $get->resume();
        }

        $contents = stream_get_contents($sockets[1]);

        if($contents !== false) {
            echo $contents;
            flush();
        }
    } catch(Exception $e) {
        fclose($sockets[0]);    // wtb finally
        fclose($sockets[1]);

        throw $e;
    }

    fclose($sockets[0]);
    fclose($sockets[1]);
}

// class FtpConnection
public function get_non_blocking($path, $stream) {
    // $this->ftp is the FTP resource returned by ftp_connect
    return new FtpNonBlockingRequest($this->ftp, $path, $stream);
}

/* TODO Error handling. */
class FtpNonBlockingRequest {
    protected $ftp = NULL;
    protected $status = NULL;

    public function __construct($ftp, $path, $stream) {
        $this->ftp = $ftp;

        $this->status = ftp_nb_fget($this->ftp, $stream, $path, FTP_BINARY);
    }

    public function is_finished() {
        return $this->status !== FTP_MOREDATA;
    }

    public function resume() {
        if($this->is_finished()) {
            throw BadMethodCallException('Cannot continue download; already finished');
        }

        $this->status = ftp_nb_continue($this->ftp);
    }
}

Answer 6

a quick search brought up php’s flush.

this article might also be of interest: http://www.net2ftp.org/forums/viewtopic.php?id=3774