Make cURL write data as it receives it

Question

I have the following php code which I found here:

function download_xml()
{
    $url = \'http://tv.sygko.net/tv.xml\';

    $ch = curl_init($url);
    $timeo         


        

Answer 1

Here comes a fully working example:

public function saveFile($url, $dest) {

        if (!file_exists($dest))
                touch($dest);

        $file = fopen($dest, 'w');
        $ch = curl_init();

        curl_setopt($ch, CURLOPT_URL, $url);
        curl_setopt($ch, CURLOPT_PROGRESSFUNCTION, 'progressCallback');
        curl_setopt($ch, CURLOPT_BUFFERSIZE, (1024*1024*512));
        curl_setopt($ch, CURLOPT_NOPROGRESS, FALSE);
        curl_setopt($ch, CURLOPT_FAILONERROR, 1);
        curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);
        curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
        curl_setopt($ch, CURLOPT_TIMEOUT, 15);
        curl_setopt($ch, CURLOPT_USERAGENT, 'Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.13) Gecko/20080311 Firefox/2.0.0.13');
        curl_setopt($ch, CURLOPT_FILE, $file);

        curl_exec($ch);
        curl_close($ch);

        fclose($file);
}
?>

The secret lies withing setting CURLOPT_NOPROGRESS to FALSE, and then, CURLOPT_BUFFERSIZE will make the callback report for every CURLOPT_BUFFERSIZE bytes reached. The smaller value, the more frequently it will report. This also depends on your download speed, etc, so don't count on it to report every X seconds, since it will report for every X bytes received/transferred.

Answer 2

curl_setopt the CURLOPT_FILE - The file that the transfer should be written to. The default is STDOUT (the browser window)

http://us2.php.net/manual/en/function.curl-setopt.php

Answer 3

There's an option called CURELOPT_FILE that allows you to specify a file handler that curl should write to. I'm pretty sure it will do "right" thing and "write" as it reads, avoiding your memory problem

$file = fopen('test.txt', 'w'); //<--------- file handler
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL,'http://example.com');
curl_setopt($ch, CURLOPT_FAILONERROR,1);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION,1);
curl_setopt($ch, CURLOPT_RETURNTRANSFER,1);
curl_setopt($ch, CURLOPT_TIMEOUT, 15);
curl_setopt($ch, CURLOPT_FILE, $file);   //<------- this is your magic line
curl_exec($ch); 
curl_close($ch);
fclose($file);

Answer 4

Your timeout is set to 5 seconds which might be too short depending on the file size of the document. Try increasing it to 10-15 just to make sure it has enough time to complete the transfer.