无法用尽更多负载按钮以显示所有标题

时间:2018-10-24 06:09:21

标签: python python-3.x selenium selenium-webdriver web-scraping

我试图一直点击位于网页底部(位于目标网页)的 more 按钮,以显示所有标题。问题是当我执行脚本时,它只单击一次然后停止。我该如何继续单击该按钮,直到没有更多选择可以单击?

Link to that website

到目前为止,这是我的脚本:

from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC

link = "https://www.newsnow.co.uk/h/Sport/Football/Championship/Transfer+News"

driver = webdriver.Chrome()
wait = WebDriverWait(driver, 10)
driver.get(link)

while True:
    try:
        loadmore = wait.until(EC.visibility_of_element_located((By.CSS_SELECTOR, "[class^='hl_more']")))
        driver.execute_script("arguments[0].scrollIntoView();",loadmore)
        loadmore.click()
    except Exception: break
driver.quit()

1 个答案:

答案 0 :(得分:2)

请尝试以下代码来模拟所需的行为:

header = driver.find_element_by_id("phead")
driver.execute_script('arguments[0].style.position = "absolute";', header)
while True:
    try:    
        loadmore = wait.until(EC.visibility_of_element_located((By.CSS_SELECTOR, "a.hl_more.bg_tween:not(.nfloading)")))
        driver.execute_script("arguments[0].scrollIntoView();",loadmore)
        loadmore.click()
    except: break