标签: python python-2.7 scrapy
根据此问题How Scrapy filters the crawled urls?, JOBDIR 变量
requests.seen
请问我在哪里可以找到JOBDIR变量?
答案 0 :(得分:2)
根据official tutorial(Jobs: pausing and resuming crawls),可以从命令行设置JOBDIR:
scrapy crawl somespider -s JOBDIR=crawls/somespider-1