How to make asynchronous HTTP requests in PHP

Question

Is there a way in PHP to make asynchronous HTTP calls? I don\'t care about the response, I just want to do something like file_get_contents(), but not wait for

Answer 1

I find this package quite useful and very simple: https://github.com/amphp/parallel-functions

<?php

use function Amp\ParallelFunctions\parallelMap;
use function Amp\Promise\wait;

$responses = wait(parallelMap([
    'https://google.com/',
    'https://github.com/',
    'https://stackoverflow.com/',
], function ($url) {
    return file_get_contents($url);
}));

It will load all 3 urls in parallel. You can also use class instance methods in the closure.

For example I use Laravel extension based on this package https://github.com/spatie/laravel-collection-macros#parallelmap

Here is my code:

    /**
     * Get domains with all needed data
     */
    protected function getDomainsWithdata(): Collection
    {
        return $this->opensrs->getDomains()->parallelMap(function ($domain) {
            $contact = $this->opensrs->getDomainContact($domain);
            $contact['domain'] = $domain;
            return $contact;
        }, 10);
    }

It loads all needed data in 10 parallel threads and instead of 50 secs without async it finished in just 8 secs.

Answer 2

/**
 * Asynchronously execute/include a PHP file. Does not record the output of the file anywhere. 
 *
 * @param string $filename              file to execute, relative to calling script
 * @param string $options               (optional) arguments to pass to file via the command line
 */ 
function asyncInclude($filename, $options = '') {
    exec("/path/to/php -f {$filename} {$options} >> /dev/null &");
}

Answer 3

Here is a working example, just run it and open storage.txt afterwards, to check the magical result

<?php
    function curlGet($target){
        $ch = curl_init();
        curl_setopt($ch, CURLOPT_URL, $target);
        curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
        $result = curl_exec ($ch);
        curl_close ($ch);
        return $result;
    }

    // Its the next 3 lines that do the magic
    ignore_user_abort(true);
    header("Connection: close"); header("Content-Length: 0");
    echo str_repeat("s", 100000); flush();

    $i = $_GET['i'];
    if(!is_numeric($i)) $i = 1;
    if($i > 4) exit;
    if($i == 1) file_put_contents('storage.txt', '');

    file_put_contents('storage.txt', file_get_contents('storage.txt') . time() . "\n");

    sleep(5);
    curlGet($_SERVER['HTTP_HOST'] . $_SERVER['SCRIPT_NAME'] . '?i=' . ($i + 1));
    curlGet($_SERVER['HTTP_HOST'] . $_SERVER['SCRIPT_NAME'] . '?i=' . ($i + 1));

Answer 4

You can use non-blocking sockets and one of pecl extensions for PHP:

• http://php.net/event
• http://php.net/libevent
• http://php.net/ev
• https://github.com/m4rw3r/php-libev

You can use library which gives you an abstraction layer between your code and a pecl extension: https://github.com/reactphp/event-loop

You can also use async http-client, based on the previous library: https://github.com/reactphp/http-client

See others libraries of ReactPHP: http://reactphp.org

Be careful with an asynchronous model. I recommend to see this video on youtube: http://www.youtube.com/watch?v=MWNcItWuKpI

Answer 5

Here is my own PHP function when I do POST to a specific URL of any page.... Sample: *** usage of my Function...

    <?php
        parse_str("email=myemail@ehehehahaha.com&subject=this is just a test");
        $_POST['email']=$email;
        $_POST['subject']=$subject;
        echo HTTP_POST("http://example.com/mail.php",$_POST);***

    exit;
    ?>
    <?php
    /*********HTTP POST using FSOCKOPEN **************/
    // by ArbZ

function HTTP_Post($URL,$data, $referrer="") {

    // parsing the given URL
    $URL_Info=parse_url($URL);

    // Building referrer
    if($referrer=="") // if not given use this script as referrer
        $referrer=$_SERVER["SCRIPT_URI"];

    // making string from $data
    foreach($data as $key=>$value)
        $values[]="$key=".urlencode($value);
        $data_string=implode("&",$values);

    // Find out which port is needed - if not given use standard (=80)
    if(!isset($URL_Info["port"]))
        $URL_Info["port"]=80;

    // building POST-request: HTTP_HEADERs
    $request.="POST ".$URL_Info["path"]." HTTP/1.1\n";
    $request.="Host: ".$URL_Info["host"]."\n";
    $request.="Referer: $referer\n";
    $request.="Content-type: application/x-www-form-urlencoded\n";
    $request.="Content-length: ".strlen($data_string)."\n";
    $request.="Connection: close\n";
    $request.="\n";
    $request.=$data_string."\n";

    $fp = fsockopen($URL_Info["host"],$URL_Info["port"]);
    fputs($fp, $request);
    while(!feof($fp)) {
        $result .= fgets($fp, 128);
    }
    fclose($fp); //$eco = nl2br();


    function getTextBetweenTags($string, $tagname) {
        $pattern = "/<$tagname ?.*>(.*)<\/$tagname>/";
        preg_match($pattern, $string, $matches);
        return $matches[1];
    }
    //STORE THE FETCHED CONTENTS to a VARIABLE, because its way better and fast...
    $str = $result;
    $txt = getTextBetweenTags($str, "span"); $eco = $txt;  $result = explode("&",$result);
    return $result[1];
    <span style=background-color:LightYellow;color:blue>".trim($_GET['em'])."</span>
    </pre> "; 
}
</pre>

Answer 6

let me show you my way :)

needs nodejs installed on the server

(my server sends 1000 https get request takes only 2 seconds)

url.php :

<?
$urls = array_fill(0, 100, 'http://google.com/blank.html');

function execinbackground($cmd) { 
    if (substr(php_uname(), 0, 7) == "Windows"){ 
        pclose(popen("start /B ". $cmd, "r"));  
    } 
    else { 
        exec($cmd . " > /dev/null &");   
    } 
} 
fwite(fopen("urls.txt","w"),implode("\n",$urls);
execinbackground("nodejs urlscript.js urls.txt");
// { do your work while get requests being executed.. }
?>

urlscript.js >

var https = require('https');
var url = require('url');
var http = require('http');
var fs = require('fs');
var dosya = process.argv[2];
var logdosya = 'log.txt';
var count=0;
http.globalAgent.maxSockets = 300;
https.globalAgent.maxSockets = 300;

setTimeout(timeout,100000); // maximum execution time (in ms)

function trim(string) {
    return string.replace(/^\s*|\s*$/g, '')
}

fs.readFile(process.argv[2], 'utf8', function (err, data) {
    if (err) {
        throw err;
    }
    parcala(data);
});

function parcala(data) {
    var data = data.split("\n");
    count=''+data.length+'-'+data[1];
    data.forEach(function (d) {
        req(trim(d));
    });
    /*
    fs.unlink(dosya, function d() {
        console.log('<%s> file deleted', dosya);
    });
    */
}


function req(link) {
    var linkinfo = url.parse(link);
    if (linkinfo.protocol == 'https:') {
        var options = {
        host: linkinfo.host,
        port: 443,
        path: linkinfo.path,
        method: 'GET'
    };
https.get(options, function(res) {res.on('data', function(d) {});}).on('error', function(e) {console.error(e);});
    } else {
    var options = {
        host: linkinfo.host,
        port: 80,
        path: linkinfo.path,
        method: 'GET'
    };        
http.get(options, function(res) {res.on('data', function(d) {});}).on('error', function(e) {console.error(e);});
    }
}


process.on('exit', onExit);

function onExit() {
    log();
}

function timeout()
{
console.log("i am too far gone");process.exit();
}

function log() 
{
    var fd = fs.openSync(logdosya, 'a+');
    fs.writeSync(fd, dosya + '-'+count+'\n');
    fs.closeSync(fd);
}