Question

我需要从搜索输入中搜索谷歌的建议。现在我使用selenium + phantomjs webdriver。

search_input = selenium.find_element_by_xpath(".//input[@id='lst-ib']")
search_input.send_keys('phantomjs har python')
time.sleep(1.5)
from lxml.html import fromstring
etree = fromstring(selenium.page_source)
output = []
for suggestion in etree.xpath(".//ul[@role='listbox']/li//div[@class='sbqs_c']"):
            output.append(" ".join([s.strip() for s in suggestion.xpath(".//text()") if s.strip()]))

但我在firebug XHR请求中看到this。和响应 - 简单的文本文件与我需要的数据。然后我看一下日志：

selenium.get_log("har")

我看不到这个请求。我该怎么抓住它？我需要这个url作为模板用于请求lib将其与其他搜索词一起使用。或者可能运行使用其他（而不是输入字段）参数发起此请求的js，是否可能？

Answer 1

您只能使用Python + Selenium + PhantomJS解决此问题。

以下列出了我为使其发挥作用所做的事情：

通过所需功能
使用Explicit Waits
要求直接https://www.google.com/?gws_rd=ssl#q=phantomjs+har+python网址

工作解决方案：

var f = function() {
    this.a = 4;
}
var test = new f();
alert(test.a); // 4

打印：

from selenium.webdriver.common.by import By
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium import webdriver


desired_capabilities = webdriver.DesiredCapabilities.PHANTOMJS
desired_capabilities["phantomjs.page.customHeaders.User-Agent"] = "Mozilla/5.0 (Linux; U; Android 2.3.3; en-us; LG-LU3000 Build/GRI40) AppleWebKit/533.1 (KHTML, like Gecko) Version/4.0 Mobile Safari/533.1"

driver = webdriver.PhantomJS(desired_capabilities=desired_capabilities)
driver.get("https://www.google.com/?gws_rd=ssl#q=phantomjs+har+python")

wait = WebDriverWait(driver, 10)

# focus the input and trigger the suggestion list to be shown
search_input = wait.until(EC.visibility_of_element_located((By.NAME, "q")))
search_input.send_keys(Keys.ARROW_DOWN)
search_input.click()

# wait for the suggestion box to appear
wait.until(EC.presence_of_element_located((By.CSS_SELECTOR, "ul[role=listbox]")))

# parse suggestions
print "List of suggestions: "
for suggestion in driver.find_elements_by_css_selector("ul[role=listbox] li[dir]"):
    print suggestion.text

如何使用自定义参数运行site js函数？

1 个答案: