所以,关键是我有一个蜘蛛,我想在其中传递一个ID,该ID会绑在我要抓取的网址末尾。我很难获得结果。所以问题是“我如何从蜘蛛那里得到答复?”
示例代码
import scrapy
class WarTimelineSpider(scrapy.Spider):
name = "WarTimelineSpider"
allowed_domains = ["URL HERE"]
def __init__(self, id=None, **kwargs):
super(WarTimelineSpider, self).__init__(**kwargs)
self.start_urls = (f"BASEURL/id={id}")
def start_requests(self):
yield scrapy.Request(self.start_urls, self.parse)
def parse(self, response):
name_selector = "SELECTOR HERE"
for timeline in response.css():
yield {
"stats": timeline.css(name_selector).extract_first()
}
timeline_class = WarTimelineSpider(war_id)
我尝试了
print(timeline_class)
但这给了我
<WarTimelineSpider 'WarTimelineSpider' at 0x1fc5fe21d08>
作为输出