twisted.python.failure.Failedtwisted.internet.error.ConnectionLost

时间:2018-11-29 03:48:32

标签: python scrapy web-crawler

我检索了Wikipedia的网站,并遇到如下问题。 twisted.web._newclient.ResponseNeverReceived: [<twisted.python.failure.Failure twisted.internet.error.ConnectionLost: Connection to the other side was lost in a non-clean fashion.>] [scrapy.statscollectors] INFO: Dumping Scrapy stats: {'downloader/exception_count': 3, 'downloader/exception_type_count/twisted.web._newclient.ResponseNeverReceived': 3, 'downloader/request_bytes': 930, 'downloader/request_count': 3, 'downloader/request_method_count/GET': 3, 'finish_reason': 'finished', 'finish_time': datetime.datetime(2018, 11, 29, 3, 17, 59, 308272), 'log_count/DEBUG': 4, 'log_count/ERROR': 1, 'log_count/INFO': 7, 'memusage/max': 50774016, 'memusage/startup': 50774016, 'retry/count': 2, 'retry/max_reached': 1, 'retry/reason_count/twisted.web._newclient.ResponseNeverReceived': 2, 'scheduler/dequeued': 3, 'scheduler/dequeued/memory': 3, 'scheduler/enqueued': 3, 'scheduler/enqueued/memory': 3, 'start_time': datetime.datetime(2018, 11, 29, 3, 17, 51, 524616)} 我已经浏览了一些有关此问题的网站,并且它们提供了使用不适合我的用户代理的建议。我想问这个问题其他解决方案。

0 个答案:

没有答案