Question

我无法使用任何Bellow方法访问此网站，$ url地址在我的所有浏览器中工作但是，我只是无法从该网站获取数据....这怎么可能？除了浏览器之外，甚至不能获取robots.txt（'https://www.natterer-modellbau.de/robots.txt）....

我在该网站上看到Google上的搜索结果，当我无法访问该网站时，Google如何访问该网站？

该页面在第一次尝试时拒绝了我的抓取工具，他们已经无法充分利用我的服务器IP了吗？我的脚本可以访问所有其他URL - 我很沮丧:)请帮助...

$url = 'https://www.natterer-modellbau.de/Flugzeuge';

$pageHeaders = get_headers($url,1); // DOES NOT WORK - TIMES OUT
file_get_contents($url); // DOES NOT WORK -Times OUT 

$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_USERAGENT, "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.1) Gecko/20061204 Firefox/2.0.0.1");
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, TRUE);
$res = curl_exec($ch);
$rescode = curl_getinfo($ch, CURLINFO_HTTP_CODE); 
curl_close($ch) ;
echo $res; // DOES NOT WORK - TIMES OUT

Answer 1

适合我。你可能在代理人后面。

php > $url = 'https://www.natterer-modellbau.de/Flugzeuge';
php >
php > $pageHeaders = get_headers($url,1); // DOES NOT WORK - TIMES OUT
php > file_get_contents($url); // DOES NOT WORK -Times OUT
php >
php > $ch = curl_init();
php > curl_setopt($ch, CURLOPT_URL, $url);
php > curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
php > curl_setopt($ch, CURLOPT_USERAGENT, "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.1) Gecko/20061204 Firefox/2.0.0.1");
php > curl_setopt($ch, CURLOPT_FOLLOWLOCATION, TRUE);
php > $res = curl_exec($ch);
php > $rescode = curl_getinfo($ch, CURLINFO_HTTP_CODE);
php > curl_close($ch) ;
php > echo $res;
<!DOCTYPE html>
<html lang="de">
<head>

        <meta http-equiv="content-type" content="text/html; charset=iso-8859-1">
        <meta name="description" content="Elektro, Segler/E-Segler, Verbrenner">
        <meta name="keywords" content="Elektro, Segler/E-Segler, Verbrenner">

....省略了许多行

最终结束了html

        </script>

    <script>
        jtl.load(["asset/plugin_js_head?v=4.05","asset/jtl3.js?v=4.05","asset/plugin_js_body?v=4.05",]);
            </script>

</body>
</html>
php >

cURL / PHP无法获取此信息 - 但所有浏览器都可以吗？

1 个答案: