硒单击当前页面后如何获得草率响应?

时间:2018-11-27 05:03:05

标签: python-3.x selenium scrapy

我试图单击一个带到新窗口的按钮,该窗口中存在实际数据(调用AJAX)。如何从当前页面获取响应数据?如何使用response.css提取数据?预先感谢。

import scrapy
from selenium import webdriver
import os
import time
from scrapy.http import TextResponse


class mySpider(scrapy.Spider):
    name = 'myprogram'
    allowed_domains = ['example.com']
    start_urls = ['https://example.com/def.aspx']

    chromedriver = "/home/ubuntu/chromedriver"
    os.environ["webdriver.chrome.driver"] = chromedriver
    driver = webdriver.Chrome(chromedriver)

    def parse(self, response):
        self.driver.get(response.url)
        time.sleep(5)

        next = self.driver.find_element_by_xpath('//*[@id="btnGetAttr"]')
        next.click()
        time.sleep(5)

        self.driver.switch_to.window(self.driver.window_handles[1])

        # convert current_url data to response in scrapy ????

0 个答案:

没有答案