Scrapy / Splash Select选项

时间:2018-04-16 15:39:50

标签: scrapy scrapy-spider splash scrapy-splash

我真的开始对scrapy splash的工作方式感到沮丧。我不明白如何选择值并在选择值后截取屏幕截图。到目前为止,我的代码不起作用....

抓取网址:https://www.chewy.com/kong-classic-dog-toy-large/dp/38485

选择了

,但我想在这种情况下选择并在之后截取屏幕截图。

import scrapy
from scrapy_splash import SplashRequest, SplashFormRequest
from scrapy import FormRequest, Selector


class QuotesSpider(scrapy.Spider):
    name = "quotes"

    def start_requests(self):
        urls = [
            'https://www.chewy.com/kong-classic-dog-toy-large/dp/38485',
        ]
        for url in urls:
            yield scrapy.Request(url=url, callback=self.parse)

    # Gets HTML and saves to file
    def parse(self, response):
        page = response.url.split("/")[-2]
        filename = 'quotes-%s.html' % page
        with open(filename, 'wb') as f:
            f.write(response.body)
        self.log('Saved file %s' % filename)

        yield FormRequest.from_response(
            response,
            formname='sku-selector',
            formdata={
                '15949': 'Small', 
            }
        )

        with open('request.png', 'wb') as f:
            f.write(response.body)

0 个答案:

没有答案