如何为https网站选择CURL代理?

时间:2018-10-29 19:53:47

标签: php curl web-scraping https proxy

我正在尝试使用CURL上的localhost连接到一个网站,我尝试了很多代理,但其中大多数没有用。

代码如下:

$url= 'https://stubhub.com';
$curl = curl_init();
curl_setopt( $curl, CURLOPT_URL, $url );
curl_setopt($curl, CURLOPT_REFERER, $url);
curl_setopt( $curl, CURLOPT_AUTOREFERER, TRUE );
curl_setopt( $curl, CURLOPT_HEADER, FALSE );
curl_setopt( $curl, CURLOPT_CONNECTTIMEOUT, 0 );
curl_setopt( $curl, CURLOPT_TIMEOUT, 0 );
curl_setopt( $curl, CURLOPT_RETURNTRANSFER, TRUE );
curl_setopt( $curl, CURLOPT_FOLLOWLOCATION, TRUE );
curl_setopt($curl, CURLOPT_HTTP_VERSION, CURL_HTTP_VERSION_1_1);

//certificate
$cacert='C:/xampp/cacert.pem';
curl_setopt( $curl, CURLOPT_CAINFO, $cacert );

//SSL
curl_setopt($curl, CURLOPT_SSL_VERIFYHOST, FALSE);
curl_setopt($curl, CURLOPT_SSL_VERIFYPEER, FALSE);

//Cookies
curl_setopt($curl, CURLOPT_COOKIEFILE,__DIR__."/cookie.txt");

//User-Agent
curl_setopt( $curl, CURLOPT_USERAGENT, 'Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/58.0.3029.110 Safari/537.36' );

//Proxy
$proxy = '50.115.194.97:8080';
curl_setopt($curl, CURLOPT_PROXY, $proxy);
curl_setopt($curl, CURLOPT_PROXYPORT, 443);
curl_setopt($curl, CURLOPT_HTTPPROXYTUNNEL, TRUE);


//Errors
curl_setopt($curl, CURLOPT_VERBOSE, TRUE);

$data = curl_exec( $curl );
$info = curl_getinfo( $curl );
$error = curl_error( $curl );
curl_close( $curl );

$allInfo = array($data, Info, $error);
echo '<pre>';
print_r($allInfo);
echo '</pre>';

我收到以下答复:

Array
(
    [0] => 
    [1] => Array
        (
            [url] => https://stubhub.com/
            [content_type] => 
            [http_code] => 0
            [header_size] => 0
            [request_size] => 0
            [filetime] => -1
            [ssl_verify_result] => 0
            [redirect_count] => 0
            [total_time] => 20.203
            [namelookup_time] => 0
            [connect_time] => 0
            [pretransfer_time] => 0
            [size_upload] => 0
            [size_download] => 0
            [speed_download] => 0
            [speed_upload] => 0
            [download_content_length] => -1
            [upload_content_length] => -1
            [starttransfer_time] => 0
            [redirect_time] => 0
            [redirect_url] => 
            [primary_ip] => 
            [certinfo] => Array
                (
                )

            [primary_port] => 0
            [local_ip] => 
            [local_port] => 0
        )

    [2] => Failed to connect to 50.115.194.97 port 8080: Timed out
)

我尝试使用支持https,cookie和带有443端口的代理。

是否有用于此目的的API /库?

1 个答案:

答案 0 :(得分:0)

您可以使用任何开放式代理api。

例如: gimmeproxy.com/api/getProxy?post=true&supportsHttps=true&maxCheckPeriod=7200&protocol=http

您尝试使用的代理似乎已关闭(无法正常工作)。

对于我来说,我为我们的项目构建了代理解析器和聚合器。它从一堆打开的代理列表中获取代理,将它们保存到数据库中并在循环中进行检查。 对于我们来说,它成了一个非常有用的工具。