Question

当我从终端运行我的蜘蛛时：

scrapy crawl GeneralSpider --set JOBDIR=will

一切正常，我可以看到JOBDIR文件夹

然而，当我尝试以编程方式设置它时，像这样：

from scrapy.utils.project import get_project_settings
        myS = get_project_settings()
        myS.set(myS, 'JOBDIR', "folder")

我收到以下错误：

Unhandled error in Deferred:
CRITICAL:twisted:Unhandled error in Deferred:
2016-02-15 17:49:35 [twisted] CRITICAL: Unhandled error in Deferred:


Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/scrapy/cmdline.py", line 150, in _run_command
    cmd.run(args, opts)
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/scrapy/commands/crawl.py", line 57, in run
    self.crawler_process.crawl(spname, **opts.spargs)
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/scrapy/crawler.py", line 153, in crawl
    d = crawler.crawl(*args, **kwargs)
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/twisted/internet/defer.py", line 1274, in unwindGenerator
    return _inlineCallbacks(None, gen, Deferred())
--- <exception caught here> ---
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/twisted/internet/defer.py", line 1128, in _inlineCallbacks
    result = g.send(result)
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/scrapy/crawler.py", line 70, in crawl
    self.spider = self._create_spider(*args, **kwargs)
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/scrapy/crawler.py", line 80, in _create_spider
    return self.spidercls.from_crawler(self, *args, **kwargs)
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/scrapy/spiders/__init__.py", line 50, in from_crawler
    spider = cls(*args, **kwargs)
  File "/bla bla bla spider.py", line 47, in __init__
    myS.set(myS, 'JOBDIR', "myfolder")
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/scrapy/settings/__init__.py", line 94, in set
    priority = SETTINGS_PRIORITIES[priority]
exceptions.KeyError: 'myfolder'
CRITICAL:twisted:
2016-02-15 17:49:35 [twisted] CRITICAL:

Answer 1

Settings API set method有这个签名：

set（name，value，priority ='project'）

您将设置对象作为设置名称传递，并将“myfolder”作为优先级传递，这是不期望的。

尝试这样做：

Call

scrapy如何从代码中设置JOBDIR而不是cmd

1 个答案: