Scrapyd Deploy错误:EOFError:读取一行时的EOF

时间:2017-11-09 12:59:09

标签: python ubuntu scrapy web-crawler scrapyd

我有2台电脑,PC A没有错误,爬虫已成功部署,但在PC B上,错误发生了。

我的Scrapyd服务器正在运行但是当我尝试部署我的抓取工具时,会发生这些错误。

{"status": "error", "message":Traceback (most recent call last):\\n File \"/usr/lib/python2.7/runpy.py\", line 162, in _run_module_as_main\\n \"__main__\", fname, loader, pkg_name)\\n File \"/usr/lib/python2.7/runpy.py\", line 72, in _run_code\\n exec code in run_globals\\n File \"/usr/local/lib/python2.7/dist-packages/scrapyd/runner.py\", line 40, in <module>\\n main()\\n File \"/usr/local/lib/python2.7/dist-packages/scrapyd/runner.py\", line 37, in main\\n execute()\\n File \"/usr/local/lib/python2.7/dist-packages/scrapy/cmdline.py\", line 148, in execute\\n cmd.crawler_process = CrawlerProcess(settings)\\n File \"/usr/local/lib/python2.7/dist-packages/scrapy/crawler.py\", line 243, in __init__\\n super(CrawlerProcess, self).__init__(settings)\\n File \"/usr/local/lib/python2.7/dist-packages/scrapy/crawler.py\", line 134, in __init__\\n self.spider_loader = _get_spider_loader(settings)\\n File \"/usr/local/lib/python2.7/dist-packages/scrapy/crawler.py\", line 330, in _get_spider_loader\\n return loader_cls.from_settings(settings.frozencopy())\\n File \"/usr/local/lib/python2.7/dist-packages/scrapy/spiderloader.py\", line 61, in from_settings\\n return cls(settings)\\n File \"/usr/local/lib/python2.7/dist-packages/scrapy/spiderloader.py\", line 25, in __init__\\n self._load_all_spiders()\\n File \"/usr/local/lib/python2.7/dist-packages/scrapy/spiderloader.py\", line 47, in _load_all_spiders\\n for module in walk_modules(name):\\n File \"/usr/local/lib/python2.7/dist-packages/scrapy/utils/misc.py\", line 71, in walk_modules\\n submod = import_module(fullpath)\\n File \"/usr/lib/python2.7/importlib/__init__.py\", line 37, in import_module\\n __import__(name)\\n File \"spiderman/spiders/scraper.py\", line 16, in <module>\\n mail = input('Email : ')\\nEOFError: EOF when reading a line\\n", "node_name": "MY PC"}

1 个答案:

答案 0 :(得分:0)

您需要安装以下软件包:

  • 获取更新
  • 易于安装gcc
  • apt-get install python-dev
  • pip install scrapyd
  • pip install scrapyd-client
  • pip安装beautifulsoup4
  • pip安装dateparser

然后尝试再次部署