Question

我已经在python中编写了一个与selenium结合使用的脚本，以便从启用了javascript的网页中获取一些数据。在单击下一页按钮之前，有三件事要做，因为只有当网页包含搜索结果时，才会显示下一页链接。这三件事是：填写两个搜索框并单击搜索按钮。但是，我的脚本可以完美地完成这三件事，但是当它应该点击下一页链接时会中断（抛出超时异常）。正如您所看到的，我已经尝试过三种不同的选项来点击下一页链接但从未成功过。我试过的其余两个被评论出来了。怎么做才能成功点击下一页按钮？

我尝试过的脚本：

from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC

driver = webdriver.Chrome()
wait = WebDriverWait(driver, 10)
driver.get("https://brokercheck.finra.org/")

wait.until(EC.presence_of_element_located((By.CSS_SELECTOR, "[placeholder='Name or CRD#']"))).send_keys("Michael John")
wait.until(EC.presence_of_element_located((By.CSS_SELECTOR, "[placeholder='Firm Name or CRD# (optional)']"))).send_keys("Morgan Stanley")
wait.until(EC.presence_of_element_located((By.CSS_SELECTOR,'.md-button'))).click()
# wait.until(EC.element_to_be_clickable((By.CSS_SELECTOR,'.pagination-next a'))).click()
# wait.until(EC.presence_of_element_located((By.CSS_SELECTOR,'.pagination-next a'))).click()
wait.until(EC.visibility_of_element_located((By.CSS_SELECTOR,'.pagination-next a'))).click()
driver.quit()

下一页链接所在的元素：

<ul class="pagination ng-pristine ng-untouched ng-valid ng-scope ng-isolate-scope" data-ng-if="listCtrl.getTotalResults()" total-items="listCtrl.getDisplayResults()" ng-model="listCtrl.currentPage" max-size="1" page-label="listCtrl.pageLabel($page)" items-per-page="listCtrl.itemsPerPage" ng-change="listCtrl.pageChanged()" boundary-links="true" previous-text="‹" next-text="›" first-text="«" last-text="»" aria-invalid="false">
  <!-- ngIf: ::boundaryLinks --><li ng-if="::boundaryLinks" ng-class="{disabled: noPrevious()||ngDisabled}" class="pagination-first ng-scope disabled"><a href="" ng-click="selectPage(1, $event)" class="ng-binding">«</a></li><!-- end ngIf: ::boundaryLinks -->
  <!-- ngIf: ::directionLinks --><li ng-if="::directionLinks" ng-class="{disabled: noPrevious()||ngDisabled}" class="pagination-prev ng-scope disabled"><a href="" ng-click="selectPage(page - 1, $event)" class="ng-binding">‹</a></li><!-- end ngIf: ::directionLinks -->
  <!-- ngRepeat: page in pages track by $index --><li ng-repeat="page in pages track by $index" ng-class="{active: page.active,disabled: ngDisabled&amp;&amp;!page.active}" class="pagination-page ng-scope active"><a href="" ng-click="selectPage(page.number, $event)" class="ng-binding">1 of 27 pages</a></li><!-- end ngRepeat: page in pages track by $index -->
  <!-- ngIf: ::directionLinks --><li ng-if="::directionLinks" ng-class="{disabled: noNext()||ngDisabled}" class="pagination-next ng-scope"><a href="" ng-click="selectPage(page + 1, $event)" class="ng-binding">›</a></li><!-- end ngIf: ::directionLinks -->
  <!-- ngIf: ::boundaryLinks --><li ng-if="::boundaryLinks" ng-class="{disabled: noNext()||ngDisabled}" class="pagination-last ng-scope"><a href="" ng-click="selectPage(totalPages, $event)" class="ng-binding">»</a></li><!-- end ngIf: ::boundaryLinks -->
</ul>

Answer 1

页面上有2个具有相同定位符的分页：顶部和底部。

要处理顶部，您需要执行driver.maximize_window()以使其可见，然后使用您尝试的相同代码：

link = wait.until(EC.visibility_of_element_located((By.CSS_SELECTOR,'.pagination-next a')))
driver.execute_script('arguments[0].scrollIntoView();', link)
link.click()

处理底层分页：

wait.until(EC.visibility_of_element_located((By.XPATH,'(//*[contains(@class, "pagination-next")]//a)[2]'))).click()

无法处理下一页链接

1 个答案: