Question

嗨，这是我的代码，用于扫描scrapy中的多个图像，但它只会丢弃一个图像。

你能告诉我我错在哪里吗？

def parse(self, response):
    item = DmozItem()
    image_urls = response.xpath('//div[@class="overhid"]//img/@src').extract()
    item['image_urls'] = [ x for x in image_urls]
    return item

Answer 1

问题是其他图片有lazysrc属性而不是src。获取这两个属性：

$ scrapy shell http://www.snapdeal.com/product/xolo-win-q900s-black/770747222
>>> for image in response.xpath('//div[@class="overhid"]//img'):
...     print image.xpath('@src | @lazysrc').extract()[0]
... 
http://n4.sdlcdn.com/imgs/a/k/9/small/Xolo-WIN-Q900s-Black-SDL051074306-1-9dbe9.jpg
http://n1.sdlcdn.com/imgs/a/k/9/small/Xolo-WIN-Q900s-Black-SDL051074306-2-1c8f7.jpg
http://n3.sdlcdn.com/imgs/a/k/9/small/Xolo-WIN-Q900s-Black-SDL051074306-3-09694.jpg
http://n4.sdlcdn.com/imgs/a/k/9/small/Xolo-WIN-Q900s-Black-SDL051074306-4-af867.jpg
http://n4.sdlcdn.com/imgs/a/k/9/small/Xolo-WIN-Q900s-Black-SDL051074306-5-73467.jpg
http://n2.sdlcdn.com/imgs/a/k/9/small/Xolo-WIN-Q900s-Black-SDL051074306-6-5c97f.jpg

以下是您应该如何更改parse()回调：

def parse(self, response):
    item = DmozItem()
    images = response.xpath('//div[@class="overhid"]//img')
    item['image_urls'] = [image.xpath('@src | @lazysrc').extract()[0] 
                          for image in images]
    return item

通过scrapy在单个页面中下载多个图像

1 个答案: