Question

我无法弄清楚如何访问parse函数中的对象。我想创建一个对象Check，它需要创建Product个对象。 products属性是urls的来源对象列表。

class GenericScraper(scrapy.Spider):
    name = 'will_be_overriden'
    custom_settings = {'CONCURRENT_REQUESTS': 32,
                       'DOWNLOAD_DELAY':0.5}
    def __init__(self, occs):
        super(GenericScraper,self).__init__()
        self.name = products[0].site.name
        self.products = products
        self.xpath = self.product[0].site.xpaths.first().xpath

    def start_requests(self):
        for product in self.products:
            yield scrapy.Request(url=product.url, callback=self.parse)

    def parse(self, response):
        hxs = HtmlXPathSelector(response)
        text = hxs.select(self.xpath+'/text()').extract()
        Check.objects.create(text=text,product=product) # CAN'T ACCESS CURRENT PRODUCT

        responselog.debug(response)

有可能吗？

Answer 1

使用Request meta attribute在回调之间进行通信。我假设您要将product对象与您所做的每个请求相关联，例如：

def start_requests(self):
    for product in self.products:
        yield scrapy.Request(
            url=product.url, 
            callback=self.parse, 
            meta={'product': product},
        )

def parse(self, response):
    current_product = response.meta['product']
    ...

如何在parse方法中访问对象？

1 个答案: