无法从网页

时间:2018-06-15 21:29:24

标签: python python-3.x selenium selenium-webdriver web-scraping

我编写了Python 3脚本,它使用Selenium从公鸡资源IFrame中的表中提取数据。此表包含MLB Schedule for 2018

但是,执行脚本时,我收到以下错误:

selenium.common.exceptions.TimeoutException:

当它到达我脚本中包含iframe的行时。为什么会这样?

from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC

driver = webdriver.Chrome()
driver.get("link above")
wait = WebDriverWait(driver, 10)
wait.until(EC.frame_to_be_available_and_switch_to_it(wait.until(EC.visibility_of_element_located((By.CSS_SELECTOR, "iframe#pageswitcher-content")))))
for items in wait.until(EC.presence_of_all_elements_located((By.CSS_SELECTOR, "table.waffle tr"))):
    data = [item.text for item in items.find_element_by_css_selector("td")]
    print(data)

driver.quit()

顺便说一句,如果您浏览上面的链接,您可以看到包含不同彩色徽标和文本的表格

仅供参考,我不想重新使用iframe中的链接;相反,我想切换到它来获取数据。

1 个答案:

答案 0 :(得分:1)

该页面中有两个嵌套的iframe来访问内容。试试这个:

from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC

driver = webdriver.Chrome()
driver.get("above link")
wait = WebDriverWait(driver, 10)
wait.until(EC.frame_to_be_available_and_switch_to_it(wait.until(EC.visibility_of_element_located((By.CSS_SELECTOR, "iframe")))))
wait.until(EC.frame_to_be_available_and_switch_to_it(wait.until(EC.visibility_of_element_located((By.CSS_SELECTOR, "iframe#pageswitcher-content")))))
for items in wait.until(EC.presence_of_all_elements_located((By.CSS_SELECTOR, "table.waffle tr"))):
    data = [item.text for item in items.find_elements_by_css_selector("td")]
    print(data)

driver.quit()