我创建了几个我打算与scrapyd同时运行的网络蜘蛛。我首先使用以下命令在Ubuntu 14.04中成功安装了scrapyd: pip install scrapyd,当我运行命令:scrapyd时,我在终端中得到以下输出:
2015-07-14 01:22:02-0400 [-] Log opened.
2015-07-14 01:22:02-0400 [-] twistd 13.2.0 (/usr/bin/python 2.7.6) starting up.
2015-07-14 01:22:02-0400 [-] reactor class: twisted.internet.epollreactor.EPollReactor.
2015-07-14 01:22:02-0400 [-] Site starting on 6800
2015-07-14 01:22:02-0400 [-] Starting factory <twisted.web.server.Site instance at 0x7f762f4391b8>
2015-07-14 01:22:02-0400 [Launcher] Scrapyd 1.1.0 started: max_proc=8, runner='scrapyd.runner'
我相信我得到这个输出的事实表明scrapy正在发挥作用;但是,当我在docs中运行命令:scrapyd-deploy时,我收到错误:找不到scrapyd-deploy:命令。如果安装成功,这怎么可能?我在配置文件中包含以下目标:
[deploy:scrapyd2]
url = http://scrapyd.mydomain.com/api/scrapyd/
username = name
password = secret
我不完全确定目标是如何工作的,但我基本上是从文档中复制它,所以我认为它会起作用。有什么东西我应该导入或配置我没有?感谢。
答案 0 :(得分:13)
scrapyd-deploy
是scrapyd-client的一部分。您可以从PyPi安装它。尝试:
$ sudo pip install scrapyd-client
答案 1 :(得分:0)
可以直接从git安装:
ospackage{
// (...)
into("/modules"){
into ("properties"){
from ("/SOURCES/Properties") // you could add some filtering
}
into ("binaries"){
from ("/SOURCES/war")
}
into ("conf"){
from ("/SOURCES/configs")
}
// EDIT : include all .java source files
into ("sources"){
from ("/SOURCES") {
include "**/*.java"
}
}
}
}