Question

我正在尝试将连接到django项目的scrapy部署到scrapyd，但是当我尝试scrapyd-deploy JD -p JDSpider时，它失败了。它说No module named GradutionProject。似乎scrapyd无法在 settings.py 中检测到“ GradutionProject.settings”。我尝试了路径的每种组合，但均失败了。您能告诉我可能的解决方法吗？预先感谢。

2019-03-24T01:02:02+0800 [_GenericHTTPChannelProtocol,12,127.0.0.1] Unhandled Error
        Traceback (most recent call last):
          File "d:\anacaonda\lib\site-packages\twisted\web\http.py", line 2190, in allContentReceived
            req.requestReceived(command, path, version)
          File "d:\anacaonda\lib\site-packages\twisted\web\http.py", line 917, in requestReceived
            self.process()
          File "d:\anacaonda\lib\site-packages\twisted\web\server.py", line 199, in process
            self.render(resrc)
          File "d:\anacaonda\lib\site-packages\twisted\web\server.py", line 259, in render
            body = resrc.render(self)
        --- <exception caught here> ---
          File "d:\anacaonda\lib\site-packages\scrapyd\webservice.py", line 21, in render
            return JsonResource.render(self, txrequest).encode('utf-8')
          File "d:\anacaonda\lib\site-packages\scrapyd\utils.py", line 20, in render
            r = resource.Resource.render(self, txrequest)
          File "d:\anacaonda\lib\site-packages\twisted\web\resource.py", line 250, in render
            return m(request)
          File "d:\anacaonda\lib\site-packages\scrapyd\webservice.py", line 86, in render_POST
            spiders = get_spider_list(project, version=version)
          File "d:\anacaonda\lib\site-packages\scrapyd\utils.py", line 137, in get_spider_list
            raise RuntimeError(msg.encode('unicode_escape') if six.PY2 else msg)
        builtins.RuntimeError: Traceback (most recent call last):
          File "d:\anacaonda\lib\runpy.py", line 193, in _run_module_as_main
            "__main__", mod_spec)
          File "d:\anacaonda\lib\runpy.py", line 85, in _run_code
            exec(code, run_globals)
          File "d:\anacaonda\lib\site-packages\scrapyd\runner.py", line 40, in <module>
            main()
          File "d:\anacaonda\lib\site-packages\scrapyd\runner.py", line 37, in main
            execute()
          File "d:\anacaonda\lib\site-packages\scrapy\cmdline.py", line 108, in execute
            settings = get_project_settings()
          File "d:\anacaonda\lib\site-packages\scrapy\utils\project.py", line 68, in get_project_settings
            settings.setmodule(settings_module_path, priority='project')
          File "d:\anacaonda\lib\site-packages\scrapy\settings\__init__.py", line 292, in setmodule
            module = import_module(module)
          File "d:\anacaonda\lib\importlib\__init__.py", line 126, in import_module
            return _bootstrap._gcd_import(name[level:], package, level)
          File "<frozen importlib._bootstrap>", line 978, in _gcd_import
          File "<frozen importlib._bootstrap>", line 961, in _find_and_load
          File "<frozen importlib._bootstrap>", line 950, in _find_and_load_unlocked
          File "<frozen importlib._bootstrap>", line 646, in _load_unlocked
          File "<frozen importlib._bootstrap>", line 616, in _load_backward_compatible
          File "c:\users\柘宇\appdata\local\temp_~1\jdspider-1553360516-l5i44at6.egg\JDSpider\settings.py", line 24, in <module>
          File "d:\anacaonda\lib\site-packages\django\__init__.py", line 19, in setup
            configure_logging(settings.LOGGING_CONFIG, settings.LOGGING)
          File "d:\anacaonda\lib\site-packages\django\conf\__init__.py", line 56, in __getattr__
            self._setup(name)
          File "d:\anacaonda\lib\site-packages\django\conf\__init__.py", line 43, in _setup
            self._wrapped = Settings(settings_module)
          File "d:\anacaonda\lib\site-packages\django\conf\__init__.py", line 106, in __init__
            mod = importlib.import_module(self.SETTINGS_MODULE)
          File "d:\anacaonda\lib\importlib\__init__.py", line 126, in import_module
            return _bootstrap._gcd_import(name[level:], package, level)
          File "<frozen importlib._bootstrap>", line 978, in _gcd_import
          File "<frozen importlib._bootstrap>", line 961, in _find_and_load
          File "<frozen importlib._bootstrap>", line 936, in _find_and_load_unlocked
          File "<frozen importlib._bootstrap>", line 205, in _call_with_frames_removed
          File "<frozen importlib._bootstrap>", line 978, in _gcd_import
          File "<frozen importlib._bootstrap>", line 961, in _find_and_load
          File "<frozen importlib._bootstrap>", line 948, in _find_and_load_unlocked
        ModuleNotFoundError: No module named 'GraduationProject'

scrapy.cfg

[settings]
default = JDSpider.settings

[deploy:JD]
url = http://localhost:6800/
project = JDSpider

settings.py

import os
import sys
sys.path.append(os.path.dirname(os.path.abspath('.')))
# Do not forget the change iCrawler part based on your project name
os.environ.setdefault("DJANGO_SETTINGS_MODULE", "GraduationProject.settings")

import django
django.setup()

项目层次结构

Answer 1

我今天遇到了这个问题，发现，您需要在django项目中CD到您scrapy项目的根目录，然后启动scrapyd。

这是我今天发现的内容，此回购的自述文件很有帮助：Scrapyd-Django-Template

无法将scrapy部署到scrapyd服务器

1 个答案: