Scrapy Shell response.css返回一个空数组

时间:2019-06-17 13:47:48

标签: python shell web-scraping scrapy

所以最近我开始在scrapy docs网站上学习scrapy。我尝试了一些网站。 现在,如果我想使用Scrapy Shell,那么大多数情况下我会得到空数组。

import scrapy

class rebuySpider(scrapy.Spider):
    name = "rebuy"
    allowed_domains = ['rebuy.de']

    start_urls = [
        'https://www.rebuy.de/kaufen/handy',
        'https://www.rebuy.de/kaufen/handy?page=2'
    ]

    def parse(self, response):
        page = response.url.split("=")[-1]
        filename = 'rebuy-%s.html' % page
        with open(filename, 'wb') as f:
            f.write(response.body)

    def parse(self, response):
        for product in response.css("div.ry-product__item"):
            yield {
                "product" : product.css("div.ry-product-item-content__name::text").getall(),
                "preis" : product.css("strong.ry-price__amount::text").getall(),
            }

0 个答案:

没有答案