Having trouble limiting download size of PHP's cURL function

Question

I\'m using PHP\'s cURL function to read profiles from steampowered.com. The data retrieved is XML, and only the first roughly 1000 bytes are needed.

The method I\'m

Answer 1

The server does not honor the Range header. The best you can do is to cancel the connection as soon as you receive more data than you want. Example:

<?php
$curl_url = 'http://steamcommunity.com/id/edgen?xml=1';
$curl_handle = curl_init($curl_url);

$data_string = "";
function write_function($handle, $data) {
    global $data_string;
    $data_string .= $data;
    if (strlen($data_string) > 1000) {
        return 0;
    }
    else
        return strlen($data);
}

curl_setopt ($curl_handle, CURLOPT_RETURNTRANSFER, 1);
curl_setopt ($curl_handle, CURLOPT_CONNECTTIMEOUT, 2);
curl_setopt ($curl_handle, CURLOPT_WRITEFUNCTION, 'write_function');

curl_exec($curl_handle);

echo $data_string;

Perhaps more cleanly, you could use the http wrapper (this would also use curl if it was compiled with --with-curlwrappers). Basically you would call fread in a loop and then fclose on the stream when you got more data than you wanted. You could also use a transport stream (open the stream with fsockopen, instead of fopen and send the headers manually) if allow_url_fopen is disabled.