批处理作业结束后触发胶水作业

时间:2020-03-30 10:54:44

标签: web-scraping scrapy scrapyd

我正在尝试向scrapyd添加爬网服务,因此我向scrapyrt.json添加了scrapyd.cfg服务,现在看起来像这样;

[services]
schedule.json     = scrapyd.webservice.Schedule
cancel.json       = scrapyd.webservice.Cancel
addversion.json   = scrapyd.webservice.AddVersion
listprojects.json = scrapyd.webservice.ListProjects
listversions.json = scrapyd.webservice.ListVersions
listspiders.json  = scrapyd.webservice.ListSpiders
delproject.json   = scrapyd.webservice.DeleteProject
delversion.json   = scrapyd.webservice.DeleteVersion
listjobs.json     = scrapyd.webservice.ListJobs
daemonstatus.json = scrapyd.webservice.DaemonStatus
scrapyrt.json =  scrapyrt.resources.CrawlResource

我希望能够调用scrapyrt(https://github.com/scrapinghub/scrapyrt)作为scrapyd的服务。 但是,当我尝试运行它时,在scrapyd中加载的项目并未在scrapyrt中加载,您是否对如何包装CrawlRessource类以使其与scrapy服务兼容具有想法?

另外,如果您还有另一种方法可以使草木和草木彼此相邻,那对我来说将非常有趣

0 个答案:

没有答案
相关问题