Question

我遇到类似下面这个人的问题。我试图使用meta属性传递一个项目。我看到输出的项目数量正确，但它们是单个项目的重复项目。有人可以帮忙吗？我猜测对之前个人帖子的反应，这应该是一个明显的解决方法。

https://github.com/scrapy/scrapy/issues/1257

def parse(self, response):
# some treatment
# a loop
     request = scrapy.Request(url=<calculated_url>, callback=parseChapter)
     request.meta['item'] = # a dictionary containing some data of the just parsed page
     yield request

def parseChapter(self, response):
# some treatment
# a loop
     request = scrapy.Request(url=<calculated_url>, callback=parseCategory)
     request.meta['item'] = # a dictionary containing some data of the just parsed page
     # print request.meta['item'] is good and different in every iteration
     yield request

def parseCategory(self, response):
# print response.meta['item'] is not good because it displays the same value many times
# for every new call of parseChapter, meta['item'] received is always the same
# some treatment

Answer 1

最有可能的是，您在for循环的每次迭代中修改项目而不是创建新项目。

因此，所有请求都以相同的值发送。即项目变量的最后一个值。

def parseChapter(self, response):
# some treatment
# a loop
     request = scrapy.Request(url=<calculated_url>, callback=parseCategory)
     request.meta['item'] = my_item_dict.copy()
     # print request.meta['item'] is good and different in every iteration
     yield request

Scrapy没有通过元属性传递正确的项目

1 个答案: