Question

我正试图屏蔽一个网站（下面的代码段）

网站接受输入，导航到第二页并获取更多输入，最后显示一个表格。我在这一步失败了：

driver.find_element_by_xpath("//select[@id='agencies']/option[@value='13156']").click()

我得到的错误是：

selenium.common.exceptions.NoSuchElementException: Message: 'Unable to locate element:

这很奇怪，因为我确实看到了元素（注释显示ID）。请问任何帮助/指示？

（我尝试过请求/ RoboBrowser - 似乎无法让帖子工作但也失败了）

from selenium import webdriver
from selenium import selenium
from bs4 import BeautifulSoup

driver = webdriver.Firefox()
url = 'http://www.ucrdatatool.gov/Search/Crime/Local/OneYearofData.cfm'
driver.get(url)

driver.find_element_by_xpath("//select[@id='state']/option[@value='1']").click()
#driver.find_element_by_xpath("//select[@id='groups']/option[@value='8']").click()

driver.find_element_by_xpath("//input[@type='submit' and @value='Next']").click()
driver.implicitly_wait(5) # seconds

# Display id tags
#elementsAll = driver.find_elements_by_xpath('//*[@id]')
#for elements in elementsAll:
#    print("id: ", repr(elements))
#    print("idName: ",elements.get_attribute("id"))
#    driver.implicitly_wait(5) # seconds

driver.find_element_by_xpath("//select[@id='groups']/option[@value='2']").click()
driver.find_element_by_xpath("//select[@id='year']/option[@value=1986]").click()
driver.find_element_by_xpath("//select[@id='agencies']/option[@value='13156']").click()

更新 - below适用于Selenium。我打算在列表框中选择所有选项并保存查询结果...感谢指针，Alecxe！

select = Select(driver.find_element_by_id('agencies'))
for options in select.options:
    select.select_by_visible_text(options.text)

select = Select(driver.find_element_by_id('groups'))
for options in select.options:
    select.select_by_visible_text(options.text)

driver.find_element_by_xpath("//select[@id='year']/option[@value=1985]").click()

driver.find_element_by_xpath("//input[@type='submit' and @value='Get Table']").click()

Answer 1

option中13156的{{1}}值select没有agencies。有102到522的值，您可以通过打印查看它们：

[element.get_attribute('value') for element in driver.find_elements_by_xpath('//select[@id="agencies"]/option')]

此外，不是按option查找value，而是使用Select并按文字获取选项：

from selenium.webdriver.support.ui import Select

select = Select(driver.find_element_by_id('agencies'))
print select.options
select.select_by_visible_text('Selma Police Dept')

Python Selenium Screen Scrape

1 个答案: