PHP CURL:在循环中爬行多个页面

时间:2014-09-04 07:12:20

标签: php curl

我试图通过循环从多个网址获取数据。在第一个循环中,我获取数据,但在此之后"没有数据收到"消息由浏览器显示。我试图延迟功能,但它不起作用。我该怎么办?我总是得到第一页,但之后我没有得到数据。这是我的代码:

public function Crawl(){

    $url=array(0=>array("ABCD"=>
        array(0=>"http://abcd.com"
        )),
        1=>array("XYZ"=>
        array(0=>"xyz.com"
        ))
        );

        foreach ($url as $key => $value) {                
            $this->Cron($value);                
            sleep(5);
        }

}


public function Cron($url=null){        

    ob_start();
    set_time_out(0);
    foreach($url as $urlkey=>$urlvalue){

         for($prodcount=0;$prodcount<count($urlvalue);$prodcount++){

         $ch = curl_init();  // Initialising cURL
        curl_setopt($ch, CURLOPT_URL, $urlvalue[$prodcount]);    // Setting cURL's URL option with the $url variable passed into the function
        curl_setopt($ch, CURLOPT_RETURNTRANSFER, TRUE); // Setting cURL's option to return the webpage data
        $data = curl_exec($ch); // Executing the cURL request and assigning the returned data to the $data variable
        curl_close($ch);    // Closing cURL
         //echo $data;   // Returning the data from the function

        $html = str_get_html($data);

        echo $html;

         ob_flush();
    }

}
}

1 个答案:

答案 0 :(得分:0)

我认为html可能不是很好的问题。

尝试添加标签,最重要的是:

<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">