Scrapy cmdline.execute会停止脚本

时间:2014-07-21 21:38:15

标签: command-line scrapy

当我打电话

cmdline.execute("scrapy crawl website".split())
print "Hello World"

它在命令line.execute之后停止脚本,并且不运行脚本的其余部分并打印“Hello World”。我该如何解决这个问题?

3 个答案:

答案 0 :(得分:1)

通过查看Scrapy execute中的cmdline.py函数,您会看到最后一行是:

sys.exit(cmd.exitcode)

如果直接调用sys.exit函数,至少在没有更改的情况下调用execute函数,则无法绕过此execute调用。猴子修补是一种选择,虽然不是一个好选择!更好的选择是避免完全调用from twisted.internet import reactor from scrapy import log, signals from scrapy.crawler import Crawler as ScrapyCrawler from scrapy.settings import Settings from scrapy.xlib.pydispatch import dispatcher from scrapy.utils.project import get_project_settings def scrapy_crawl(name): def stop_reactor(): reactor.stop() dispatcher.connect(stop_reactor, signal=signals.spider_closed) scrapy_settings = get_project_settings() crawler = ScrapyCrawler(scrapy_settings) crawler.configure() spider = crawler.spiders.create(name) crawler.crawl(spider) crawler.start() log.start() reactor.run() 函数,而是使用下面的自定义函数:

scrapy_crawl("your_crawler_name")

你可以这样称呼它:

<input type="text" class="firstName" name="firstName" data-rule-required="true" /> 

答案 1 :(得分:1)

可以运行subprocess.call。例如,在带有PowerShell的Windows上:

导入子流程

subprocess.call([R'C:\窗口\ system32 \ WindowsPowerShell \ V1.0 \ powershell.exe”,                              '-ExecutionPolicy',                              “无限制”,                  'scrapy crawl website -o items.json -t json'])

答案 2 :(得分:1)

我刚刚尝试了以下代码,它对我有用:

class Student5{  
    int id;  
    String name;  
    int age;  

public Student5(int i,String n){  
System.out.println(i+" "+n);
    id = i;  
    name = n;  
    }  

public Student5(int i,String n,int a){
this(i, n);
System.out.println(i+" "+n+" "+a);
    id = i;  
    name = n;  
    age=a;  
    }  

void display(){
     System.out.println(id+" "+name+" "+age);
   }  


public static void main(String args[]){  
//   Student5 s1 = new Student5(111,"Rick");  
    Student5 s2 = new Student5(222,"Morty",25);  
//   s1.display();
    //s2.display();

   }  
}