点击带有硒的网页中的“显示更多交易”

时间:2018-11-20 12:37:34

标签: python selenium web-scraping lazy-loading webdriverwait

我想在下一页上单击每个“显示10笔以上的交易”:“ https://www.uswitch.com/broadband/compare/deals_and_offers/”,但似乎不起作用。 我陷入了以下错误:

 AttributeError: 'NoneType' object has no attribute 'find_element'

我的代码如下:

from selenium import webdriver
from selenium.webdriver.support.wait import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.by import By

url = "https://www.uswitch.com/broadband/compare/deals_and_offers/"

driver = webdriver.Chrome(r'C:\temp\chromedriver.exe')

browser = driver.get(url)
while True:
    button = WebDriverWait(browser,10).until(EC.presence_of_element_located((By.PARTIAL_LINK_TEXT, 'Show 10 more deals')))
button.click()

有什么主意吗?

3 个答案:

答案 0 :(得分:1)

要在页面https://www.uswitch.com/broadband/compare/deals_and_offers/上单击文本为显示10个以上交易的元素,可以使用以下解决方案:

  • 代码块:

    from selenium import webdriver
    from selenium.webdriver.support.wait import WebDriverWait
    from selenium.webdriver.support import expected_conditions as EC
    from selenium.webdriver.common.by import By
    
    url = "https://www.uswitch.com/broadband/compare/deals_and_offers/"
    options = webdriver.ChromeOptions() 
    options.add_argument("start-maximized")
    options.add_argument('disable-infobars')
    browser = webdriver.Chrome(chrome_options=options, executable_path=r'C:\Utility\BrowserDrivers\chromedriver.exe')
    browser.get(url)
    while True:
        try:
            browser.execute_script("return arguments[0].scrollIntoView(true);", WebDriverWait(browser,20).until(EC.visibility_of_element_located((By.XPATH, "//button[@class='us-btn us-btn--action' and contains(.,'Show 10 more deals')]"))))
            browser.execute_script("arguments[0].click();", WebDriverWait(browser,20).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "button.us-btn.us-btn--action[name='visible_products']"))))
            print("Button clicked")
        except:
            print("No more Buttons")
            break
    browser.quit()
    
  • 控制台输出:

    Button clicked
    Button clicked
    Button clicked
    Button clicked
    Button clicked
    Button clicked
    Button clicked
    Button clicked
    Button clicked
    Button clicked
    No more Buttons
    

答案 1 :(得分:0)

尝试以下使用CSS attribute = value选择器通过按钮的值作为按钮的data-event-action属性的目标的方法

driver.find_element_by_css_selector('[data-event-action="Show 10 more products"]').click()

如果需要,将driver替换为browser

答案 2 :(得分:0)

像这样尝试:

while not re.search(r"Showing (\d+) of \1 ", browser.page_source):
  browser.execute_script("document.querySelector('[data-event-label=\"Show 10 more products\"]').click()")
  time.sleep(1)

这避免了硒错误,这些错误最终会使您发疯。