How can i scrape the google search result 2nd page only?

旧街凉风 提交于 2020-03-05 03:23:08

问题


I'm having trouble on scraping the second page of the google search result, After scraping the result it will save the page on the file. It only scrape the first page. Here is my code:

$url = 'http://www.google.com/search?q='.$in;
$datenbank = "proxy_work.php"; 
$datei = fopen($datenbank,"w+");
$datenbank = "proxy_work.php";
fwrite($datei, $url);  
fwrite ($datei,"\r\n");
fclose($datei);
// echo file_get_contents("proxy_work.php");
$html = file_get_html("proxy_work.php");
foreach($html->find('a') as $e){
    //  $title = $h3->innertext;
    $link  = $e->href;
    //  if(in_array($endomain, $approveurl)){
            // 
        }
    // if it is not a direct link but url reference found inside it, then extract
    if (!preg_match('/^https?/', $link) && preg_match('/q=(.+)&sa=/U', $link, $matches) && preg_match('/^https?/', $matches[1])) {
        $link = $matches[1];
    } else if (!preg_match('/^https?/', $link)) { // skip if it is not a valid link
        continue;
    } 
}

来源:https://stackoverflow.com/questions/59747547/how-can-i-scrape-the-google-search-result-2nd-page-only

易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!