在使用Selenium Webdriver进行分页时,最后一页会重复吗?

时间:2019-05-23 18:59:51

标签: python selenium-webdriver

我正在分页并提取每个页面的链接,但是我的问题是当我到达最后一页时,它会重复并为我提供最后一页的多个链接。

这是我的代码:

from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import Select
from bs4 import BeautifulSoup
from selenium.webdriver.support.ui import WebDriverWait
import requests
import pdb
import time
driver = webdriver.Firefox(executable_path = './geckodriver')
url = ('https://ipr.etsi.org')
driver.get(url)
button = driver.find_element_by_id(("btnConfirm")).click()
select_element = Select(driver.find_element_by_name("ctl00$cphMain$lstCompany"))
select_element.select_by_index(38) driver.find_element_by_id(("ctl00_cphMain_btnSearch")).click()
time.sleep(30)
a = driver.find_element_by_id("ctl00_cphMain_gridSearch_ctl00")
elems = a.find_elements_by_tag_name("a")
for ele in elems:
    a = (ele.get_attribute('href'))
    print(a)
while True:
    divs = driver.find_element_by_class_name('rgPageNext')
    try:
        divs.click()
        time.sleep(30)
        a = driver.find_element_by_id("ctl00_cphMain_gridSearch_ctl00")
        elem = a.find_elements_by_tag_name("a")
        for el in elem:                                                                                  
            a = (el.get_attribute('href'))
            print(a)
    except:
        break

0 个答案:

没有答案