php multiple curl urls with while loop

问题

I am developing a small script and I am bit messed up with using a couple of curl and while loop.

I want to stop processing curl at a point when one of the URL is giving me a information. Note: I have multiple curl requests.

So my concept is,

I have a couple of URLS, which I have to process and get information from. If information is found on a particular URL, it will be giving me a string. If no information is found, it will give me no value. So I have nearly 10 URLs to process approximately. In all cases, any one of the URL will be giving me information, so the remaining urls will be producing no value. Since processing there much URLS, latency is a issue. So suppose in the sample code below, if the url ends with value2.php gives me a result, then I immediately wanted to stop processing the other URLs. Because I already got the result and no point in running other curl. Then finally I have to print the result.

Also I have a condition where none of the URL produce any result and it will be great if someone shows me how to handle that also.

My sample code.

<?php
///functions here
do {

$ch = curl_init();
curl_setopt($ch, CURLOPT_URL,"example[dot]com/value1.php?process=$param");
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
$combined = curl_exec ($ch);
curl_close ($ch);

$ch = curl_init();
curl_setopt($ch, CURLOPT_URL,"example[dot]com/value2.php?process=$param");
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
$combined = curl_exec ($ch);
curl_close ($ch);


$ch = curl_init();
curl_setopt($ch, CURLOPT_URL,"example[dot]com/value3.php?process=$param");
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
$combined = curl_exec ($ch);
curl_close ($ch);


$ch = curl_init();
curl_setopt($ch, CURLOPT_URL,"example[dot]com/value4.php?process=$param");
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
$combined = curl_exec ($ch);
curl_close ($ch);


$ch = curl_init();
curl_setopt($ch, CURLOPT_URL,"example[dot]com/value5.php?process=$param");
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
$combined = curl_exec ($ch);
curl_close ($ch);


$ch = curl_init();
curl_setopt($ch, CURLOPT_URL,"example[dot]com/value6.php?process=$param");
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
$combined = curl_exec ($ch);
curl_close ($ch);


$ch = curl_init();
curl_setopt($ch, CURLOPT_URL,"example[dot]com/value7.php?process=$param");
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
$combined = curl_exec ($ch);
curl_close ($ch);


$ch = curl_init();
curl_setopt($ch, CURLOPT_URL,"example[dot]com/value8.php?process=$param");
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
$combined = curl_exec ($ch);
curl_close ($ch);


$ch = curl_init();
curl_setopt($ch, CURLOPT_URL,"example[dot]com/value9.php?process=$param");
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
$combined = curl_exec ($ch);
curl_close ($ch);

$ch = curl_init();
curl_setopt($ch, CURLOPT_URL,"example[dot]com/value10.php?process=$param");
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
$combined = curl_exec ($ch);
curl_close ($ch);

} while (strlen($combined) != 0);
echo $combied;

///functions here

?>

回答1:

Try the following:

<?php

function callCURL($url) {
    $ch = curl_init();
    curl_setopt($ch, CURLOPT_URL, $url);
    curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
    $combined = curl_exec ($ch);
    curl_close ($ch);
    return $combined;
}


function getResult($urls) {
    $return = array();

    foreach ($urls as $url) {
        $response = callCURL($url);
        if (strlen($response) !== 0) {
            $return[] = $response;
            break;
        }
    }
    return $return;
}

$urls = array("example.com/value1.php?process=$param", "example.com/value2.php?process=$param", "example.com/value3.php?process=$param")

$result = getResult($urls);

回答2:

Like you have it in your question and how the other answers have it, the problem is that for each request you have to send it, wait for a response, process the data and only then do you make your subsequent requests. This works fine, but it is highly time-inefficient. Say, for example, each request takes 100 ms to make (which probably not unrealistic). For 10 requests you're looking at 1 second of load time. Instead I would recommend forgetting about trying to stop making requests after you find your result and send all the requests... simultaneously. This can be accomplished with PHP's curl_multi_* functions.

// Put all of your URLs in here. I'm just using google for
// all as an example:
$urls[] = 'http://www.google.com';
$urls[] = 'http://www.google.com';
$urls[] = 'http://www.google.com';
$urls[] = 'http://www.google.com';
$urls[] = 'http://www.google.com';
$urls[] = 'http://www.google.com';

// Get cURL handles
foreach ($urls as $key => $url) {
    $chs[$key] = curl_init();

    // Set all your options for each connection here
    curl_setopt($chs[$key], CURLOPT_URL, $url);
    curl_setopt($chs[$key], CURLOPT_HEADER, 0);
}

//create the multiple cURL handle
$mh = curl_multi_init();

//add the handles
foreach ($chs as &$ch) {
    curl_multi_add_handle($mh,$ch);
}

$active = null;
//execute the handles
do {
    $mrc = curl_multi_exec($mh, $active);
} while ($mrc == CURLM_CALL_MULTI_PERFORM);

while ($active && $mrc == CURLM_OK) {
    if (curl_multi_select($mh) != -1) {
        do {
            $mrc = curl_multi_exec($mh, $active);
        } while ($mrc == CURLM_CALL_MULTI_PERFORM);
    }
}

foreach ($chs as $url=>&$ch) {
    $html = curl_multi_getcontent($ch);

    // [do what you want with the HTML]

    curl_multi_remove_handle($mh, $ch); // remove the handle (assuming  you are done with it);
}

curl_multi_close($mh);

回答3:

You can use something like this. I didn't check the code, after debug it should work.

function callcurl($url) 
{
  $ch = curl_init();
  curl_setopt($ch, CURLOPT_URL,"example[dot]com/value5.php?process=$param");
  curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
  $ret = curl_exec ($ch);
  curl_close ($ch);
}

$urls = array('url1', 'url2'); /// etc
$combined = ''
$cnt = 0;
do {
  $combined = callcurl($urls[$cnt++]);
} while (strlen($combined) != 0 && $cnt < count($urls)); //

print $combined;

回答4:

You can achieve this using a for loop.

$fileNames = array(
   '0' => 'valueabc',
   '1' => 'valuedef',
   [...] => [...]
);
$combined = array();
for ($i = 1; $i < 10; $i++)
{
    $ch = curl_init();
    $url = "example[dot]com/" . $fileNames[$i] . ".php?process=" . $param;
    curl_setopt($ch, CURLOPT_URL, $url);
    curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
    $combined[$i] = curl_exec ($ch);
    curl_close ($ch);
}

To see all the responses you can foreach $combined.

foreach ($combined as $value=>$response)
{
    // TODO: Work with the response
    echo "[" . $value . "]" . $response;
}

来源：https://stackoverflow.com/questions/36406463/php-multiple-curl-urls-with-while-loop