Scrapy:从刮板掉落物品

时间:2019-11-13 14:49:34

标签: scrapy

由于特定情况,我想从刮板本身拖放项目,而不是将刮板的特定滴落逻辑添加到管道中。

Scrapy具有DropItem异常,该异常是管道中的nicely handled,但是如果从刮板中引发,则会产生错误:

#...
raise DropItem('Item dropped ' + self.id())

输出:

2019-11-13 13:27:27 [scrapy.core.scraper] ERROR: Spider error processing <GET http://domain.tld/> (referer: http://domain.tld/referer)
Traceback (most recent call last):
File "/usr/local/lib/python3.7/site-packages/twisted/internet/defer.py", line 654, in _runCallbacks
current.result = callback(current.result, *args, **kw)
File "/usr/local/core/core/spiders/my_spider.py", line 46, in parse get.photos())
scrapy.exceptions.DropItem: Item dropped 35

有没有更优雅的方式来处理这种情况?

0 个答案:

没有答案