在网络抓取时,远程主机错误强制关闭现有连接

时间:2016-04-07 14:01:53

标签: python networking web-scraping urllib

我使用python beautifulsoup和urllib.request从以下网站获取天气数据 -

http://www.wunderground.com/history/airport/CYTZ/

以下是获取网页的代码

 # the new url created after each day
        theurl = "http://www.wunderground.com/history/airport/CYTZ/" + theDate + "/DailyHistory.html"
        # extract the source data for analysis
        thepage = urllib.request.urlopen(theurl)
        soup = BeautifulSoup(thepage, "html.parser")

我能够抓取我需要的数据,但是在给出少量结果后程序会意外终止并出现以下错误 -

URLError: <urlopen error [WinError 10054] An existing connection was forcibly closed by the remote host>  

每次运行程序时结果的数量都会有所不同。是否有一种特殊方法可以阻止这种情况发生并在一次运行中提取所有结果?谢谢

0 个答案:

没有答案