我正在使用脚本初始化我的蜘蛛,
def setup_crawler(domain):
#spider = GeneralSpider(attributesXMLFilePath=domain)
settings = get_project_settings()
Settings.set(settings, 'JOBDIR', "SeenFolders/bla bla bla")
Settings.set(settings, 'LOG_FILE', "LogFiles/bla bla bla.log")
Settings.set(settings, 'LOG_ENABLED', "TRUE")
Settings.set(settings, 'LOG_LEVEL', "INFO")
crawler = Crawler(spidercls = GeneralSpider ,settings = settings)
crawler.signals.connect(reactor.stop, signal=signals.spider_closed)
# crawler.configure()
crawler.crawl(domain)
#crawler.start()
setup_crawler('Websitesbla bla bla.xml')
#log.start()
reactor.run()
运行我的代码后,我可以在正确的位置看到看到的文件夹,但没有日志文件。另外,我试图从蜘蛛登录,cmd(终端)上也没有出现任何内容
加上蜘蛛的 init ,我试图打印
print("LOG_FILE ="+ str(myS['LOG_FILE']))
print("JOBDIR ="+str(myS['JOBDIR']))
但结果都是null
当我使用废料0.24时,一切都很完美,但当我更新到scrapy 1时,我开始遇到这个问题。
我正在定义我的记录器
self.myLogger = logging.getLogger(self.name)
然后我写这个
self.myLogger.error('The spider {0} with xml file = {2} starts working on {1}'.format(self.name, datetime.datetime.now(), attributesXMLFilePath))