由于特定情况,我想从刮板本身拖放项目,而不是将刮板的特定滴落逻辑添加到管道中。
Scrapy具有DropItem异常,该异常是管道中的nicely handled,但是如果从刮板中引发,则会产生错误:
#...
raise DropItem('Item dropped ' + self.id())
输出:
2019-11-13 13:27:27 [scrapy.core.scraper] ERROR: Spider error processing <GET http://domain.tld/> (referer: http://domain.tld/referer)
Traceback (most recent call last):
File "/usr/local/lib/python3.7/site-packages/twisted/internet/defer.py", line 654, in _runCallbacks
current.result = callback(current.result, *args, **kw)
File "/usr/local/core/core/spiders/my_spider.py", line 46, in parse get.photos())
scrapy.exceptions.DropItem: Item dropped 35
有没有更优雅的方式来处理这种情况?