Question

我在settings.py文件中指定了外部文件的名称，我想在启动时导入这些文件。但是当我运行它时，我收到错误：AttributeError: 'MySpider' object has no attribute 'settings'。如何在启动蜘蛛时导入？

我到目前为止的代码是：

class MySpider(CrawlSpider):

    name = 'MySpider'

    def __init__(self, *args, **kwargs):

        super(MySpider, self).__init__(*args, **kwargs)

        self.crawl_behaviour = FileOps.get_crawl_behaviour(self.settings.attributes.get('CRAWL_BEHAVIOUR'))

        self.rules = (

            Rule(LinkExtractor(allow=self.crawl_behaviour), follow=True),

        )

Answer 1

这对我有用。

from scrapy.conf import settings


self.crawl_behaviour = settings['CRAWL_BEHAVIOUR']

Answer 2

您可以这样使用：

from scrapy import Spider
from scrapy.utils.project import get_project_settings

class MySpider(Spider):
    name = "MySpider"
    allowed_domains = ["example.com"]
    start_urls = ['https://www.example.com/']

    def __init__(self, query='', *args, **kwargs):
        super(MySpider, self).__init__(*args, **kwargs)
        p_setting = get_project_settings()
        self.start_urls = ['https://www.example.com']
        self.query = query
        self.crawl_behavior = p_setting.get("CRAWL_BEHAVIOUR")

Scrapy：如何应用settings.py文件

2 个答案: