我需要在站点A上刮取一个主键元素。然后提取页面A上的链接并获取其他项目。 例如: www.example.com/site.html?db=123 www.example.com/site.html?db=123&keyFromPrioPage=Scraped_key
我尝试过这样的事情:
def start_requests(self):
urls = [
'www.example.com/site.html?db=123'
]
for url in urls:
yield scrapy.Request(url=url, callback=self.parse)
def parse(self, response):
Scraped_key = response.css("div.key")
items = scrapy.Request(url=url, callback=self.item)
def item(self, response):
return response.css("div.item")