Scrapy- exceptions.ImportError：没有名为middlewares的模块

Question

请有人帮我找到我失踪的地方，找到我的代码抛出的 exceptions.ImportError：没有名为middlewares的模块的解决方案。

我的文件夹结构是：

以下是settings.py中的 DOWNLOADER_MIDDLEWARES ：

DOWNLOADER_MIDDLEWARES = {
    'IpRotation.middlewares.MyCustomDownloaderMiddleware': 543,
    'scrapy.downloadermiddlewares.httpproxy.HttpProxyMiddleware': None,
    'IpRotation.ProxyMiddleware.ProxyMiddleware': 800,
    'scrapy.downloadermiddleware.useragent.UserAgentMiddleware' : None,
    'IpRotation.RotateUserAgentMiddleware.RotateUserAgentMiddleware':350
}

My Spider计划：

import scrapy    
class DmozSpider(scrapy.Spider):
        name = "dmoz"
        allowed_domains = ["dmoz.org"]
        start_urls = [
            "http://www.dmoz.org/Computers/Programming/Languages/Python/Books/",
            "http://www.dmoz.org/Computers/Programming/Languages/Python/Resources/"
        ]

        def parse(self, response):
            filename = response.url.split("/")[-2] + '.html'
            with open(filename, 'wb') as f:
                f.write(response.body)

我的自定义UserAgentMiddleware.py：

import logging
import random
import scrapy
from scrapy.downloadermiddlewares.useragent import UserAgentMiddleware

class RotateUserAgentMiddleware(UserAgentMiddleware):
    def __init__(self, user_agent=''):
        self.user_agent = user_agent

    def process_request(self, request, spider):
        user_agent_list = [....]
        ua = random.choice(user_agent_list)
        if ua:
            request.headers.setdefault('User-Agent', ua)   
            spider.log(
                u'User-Agent: {} {}'.format(request.headers.get('User-Agent'), request)

我的自定义IPRotationMiddleWare.py：

import random
from scrapy.downloadermiddlewares.httpproxy import HttpProxyMiddleware

class ProxyMiddleware(HttpProxyMiddleware):
    def __init__(self, proxy_ip=''):
        self.proxy_ip = proxy_ip

    def process_request(self,request,spider):
        ip = random.choice(self.proxy_list)
        if ip:
            request.meta['proxy']= ip


    proxy_list = [.......]

我无法找到名为exception的中间件的问题。

Answer 1

首先出现的DOWNLOADER_MIDDLEWARES出现错误：

'IpRotation.middlewares.MyCustomDownloaderMiddleware': 543,

这会在MyCustomDownloaderMiddleware模块中查找middlewares。如果此文件不存在（middlewares.py），则会收到错误。

确保您的设置对其他中间件也是正确的。

Answer 2

我相信答案很简单..

变化

'IpRotation.middlewares.MyCustomDownloaderMiddleware': 543,

到

'IpRotation.middleware.MyCustomDownloaderMiddleware': 543,

你的文件是middleware.py。请注意，它不是中间件 s .py