操作系统:Windows 7; Python 2.7.6使用Python GUI Shell
我正在尝试使用Python脚本抓取网站,有几位作者使用urllib和urllib2库。为了将URL的HTML内容存储在变量中,我看到了类似的方法:
import urllib2
c=urllib2.urlopen('http://en.wikipedia.org/wiki/Rocket_Internet')
contents=c.read( )
print contents
urlopen在120+秒后生成错误:
Traceback (most recent call last):
File "H:/Movie_Knowledge_Graph/crawl.py", line 4, in <module>
c=urllib2.urlopen('http://en.wikipedia.org/wiki/Rocket_Internet')
File "C:\Python27\lib\urllib2.py", line 127, in urlopen
return _opener.open(url, data, timeout)
File "C:\Python27\lib\urllib2.py", line 404, in open
response = self._open(req, data)
File "C:\Python27\lib\urllib2.py", line 422, in _open
'_open', req)
File "C:\Python27\lib\urllib2.py", line 382, in _call_chain
result = func(*args)
File "C:\Python27\lib\urllib2.py", line 1214, in http_open
return self.do_open(httplib.HTTPConnection, req)
File "C:\Python27\lib\urllib2.py", line 1184, in do_open
raise URLError(err)
URLError: <urlopen error [Errno 10060] A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond>
我知道在使用代理时我们必须相应地设置环境变量。但是,我正在使用我的家庭WiFi网络,这需要无代理。我也尝试过urllib,但它会产生同样的错误。