我想在设置中为我的一只蜘蛛更改一些参数。 我读了Scrapy的doc official 我试过这个:
class MySpider(CrawlSpider):
name = 'mySpider'
custom_settings = {
DOWNLOADER_MIDDLEWARES = {
'Crawling.middlewares.ProxyMiddleware' : 100,
'Crawling.middlewares.RandomUserAgentMiddleware' : 400,
'scrapy.dowloadermiddlewares.useragent.UserAgentMiddleware' : None,
'scrapyjs.SplashMiddleware' : 725,
},
COOKIES_ENABLED= True,
DOWNLOAD_DELAY= 0,
}
但是我收到了这个错误
DOWNLOADER_MIDDLEWARES = {
^
SyntaxError: invalid syntax
答案 0 :(得分:1)
设置名称必须是dict键,请查看doc
class MySpider(CrawlSpider):
name = 'mySpider'
custom_settings = {
'DOWNLOADER_MIDDLEWARES': {
'Crawling.middlewares.ProxyMiddleware' : 100,
'Crawling.middlewares.RandomUserAgentMiddleware' : 400,
'scrapy.dowloadermiddlewares.useragent.UserAgentMiddleware' : None,
'scrapyjs.SplashMiddleware' : 725,
},
'COOKIES_ENABLED': True,
'DOWNLOAD_DELAY': 0,
}