脚本无法继续点击“加载更多”按钮

时间:2018-09-03 21:59:03

标签: python python-3.x selenium selenium-webdriver web-scraping

我已经用Python与硒结合编写了一个脚本,以不断单击MORE按钮来加载更多项目,直到网页上没有要加载的新项目为止。但是,我下面的脚本可以单击该页面底部可用的MORE按钮一次。

Link to that site

这是我到目前为止的尝试:

from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC

link = "https://angel.co/companies?company_types[]=Startup&company_types[]=Private+Company&company_types[]=Mobile+App&locations[]=1688-United+States"

driver = webdriver.Chrome()
wait = WebDriverWait(driver, 10)
driver.get(link)
while True:
    for elems in wait.until(EC.presence_of_all_elements_located((By.CSS_SELECTOR,".results .name a.startup-link"))):
        print(elems.get_attribute("href"))

    try:
        loadmore = wait.until(EC.visibility_of_element_located((By.CSS_SELECTOR,"[class='more']")))
        driver.execute_script("arguments[0].scrollIntoView();", loadmore)
        loadmore.click()
    except Exception:break

driver.quit()

如何继续单击该MORE按钮,直到没有剩下的按钮可以单击并解析链接为止,就像我已经尝试过使用for loop一样。

2 个答案:

答案 0 :(得分:2)

我已经设法解决了我现有脚本中追求安德森爵士逻辑的问题。这就是修改后的脚本的样子。

from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC

link = "https://angel.co/companies?company_types[]=Startup&company_types[]=Private+Company&company_types[]=Mobile+App&locations[]=1688-United+States"

driver = webdriver.Chrome()
wait = WebDriverWait(driver, 10)
driver.get(link)
while True:
    try:
        loadmore = wait.until(EC.visibility_of_element_located((By.CSS_SELECTOR,"[class='more']")))
        driver.execute_script("arguments[0].click();", loadmore)
        wait.until(EC.staleness_of(loadmore))
    except Exception:break

for elems in wait.until(EC.presence_of_all_elements_located((By.CSS_SELECTOR,".results .name a.startup-link"))):
    print(elems.get_attribute("href"))

driver.quit()

答案 1 :(得分:0)

为什么不只是?

$( 'input#title' ).on( 'focus', function() {
    $(".mynote").show();
}).on( 'focusout', function() {
    $(".mynote").hide();
});

c#示例。非常确定它也可以使用python完成