如何在Scrapy中设置2个不同的设置?

时间:2016-06-02 11:14:18

标签: scrapy

我想在设置中为我的一只蜘蛛更改一些参数。 我读了Scrapy的doc official 我试过这个:

class MySpider(CrawlSpider):
    name = 'mySpider'
    custom_settings = {
        DOWNLOADER_MIDDLEWARES = {
            'Crawling.middlewares.ProxyMiddleware' : 100,
                'Crawling.middlewares.RandomUserAgentMiddleware' : 400,
                'scrapy.dowloadermiddlewares.useragent.UserAgentMiddleware' : None,
                'scrapyjs.SplashMiddleware' : 725,
        },
            COOKIES_ENABLED= True,
            DOWNLOAD_DELAY= 0,
    }

但是我收到了这个错误

    DOWNLOADER_MIDDLEWARES = {
                           ^
SyntaxError: invalid syntax

1 个答案:

答案 0 :(得分:1)

设置名称必须是dict键,请查看doc

class MySpider(CrawlSpider):
    name = 'mySpider'
    custom_settings = {
        'DOWNLOADER_MIDDLEWARES': {
            'Crawling.middlewares.ProxyMiddleware' : 100,
                'Crawling.middlewares.RandomUserAgentMiddleware' : 400,
                'scrapy.dowloadermiddlewares.useragent.UserAgentMiddleware' : None,
                'scrapyjs.SplashMiddleware' : 725,
        },
        'COOKIES_ENABLED': True,
        'DOWNLOAD_DELAY': 0,
    }