我如何从蜘蛛那里得到回应?

时间:2020-04-16 17:10:55

标签: python scrapy

所以,关键是我有一个蜘蛛,我想在其中传递一个ID,该ID会绑在我要抓取的网址末尾。我很难获得结果。所以问题是“我如何从蜘蛛那里得到答复?”

示例代码

import scrapy


class WarTimelineSpider(scrapy.Spider):
    name = "WarTimelineSpider"
    allowed_domains = ["URL HERE"]

    def __init__(self, id=None, **kwargs):
        super(WarTimelineSpider, self).__init__(**kwargs)
        self.start_urls = (f"BASEURL/id={id}")

    def start_requests(self):
        yield scrapy.Request(self.start_urls, self.parse)

    def parse(self, response):
        name_selector = "SELECTOR HERE"
        for timeline in response.css():
            yield {
                "stats": timeline.css(name_selector).extract_first()
            }


timeline_class = WarTimelineSpider(war_id)

我尝试了

print(timeline_class)

但这给了我

<WarTimelineSpider 'WarTimelineSpider' at 0x1fc5fe21d08>

作为输出

0 个答案:

没有答案