我对网页抓取还很陌生。我正在尝试用python用selenium编写一些东西，这些东西会自动登录到网站并从下拉菜单中单击多个选项。设置完所有这些选项后，将单击一个按钮，然后弹出带有多个href的新页面。这就是我遇到问题的地方。我试图单击所有的href，但是所有的href具有这种结构
<a href="WebsiteName.asp?qt=1&qa=0&ben=1&tpt=0&cl=Something&gl=1&life=1&smo=1">Export</a>
在上述HTML中，只有'life = 1'和'smo = 1'可能会更改为其他内容。
我在这里遇到的大多数其他问题，通常是带有带有类的href或类似的东西，从而使单击这些链接更加方便。
下面的代码是我到目前为止所拥有的。
import selenium,time
import os
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from bs4 import BeautifulSoup, SoupStrainer
import requests
#credentials
usernameStr = 'SomeUsername'
passwordStr = 'SomePassword'
browser = webdriver.Firefox(executable_path = r'C:\Users\Name\Downloads\geckodriver-v0.24.0-win64\geckodriver.exe')
url = 'http://somewebsite.com/something/'
browser.get(url)
username = browser.find_element_by_id('username')
username.send_keys(usernameStr)
password = browser.find_element_by_id('password')
password.send_keys(passwordStr)
loginInButton = browser.find_element_by_id("login")
loginInButton.click()
browser.find_element_by_xpath("//*[@id='LifeType']").send_keys("Dual")
browser.find_element_by_id("btnRefresh").click()
browser.find_element_by_id("btnExport").click()
other_url = 'http://somewebsite.com/something/exportToExcelChoice.asp?qt=1&qa=0&ben=1&tpt=0&gl=1&cl=CAESFFHIILNI'
page = requests.get(other_url)
data = page.text
soup = BeautifulSoup(data, features="html.parser")
for link in soup.find_all('a'):
link.get('href')
browser.find_element_by_link_text("Export").click()
使用Beautiful Soup，我可以轻松地打印出所需的链接，但是我要确保甚至有必要，因为我无法单击链接。我仍在努力解决这一问题。
PS，我知道这并不是严格意义上的Web抓取，因为我要做的只是单击按钮，最终目的是将所有内容都放入csv文件中。
HTML：
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.1//EN" "http://www.w3.org/TR/xhtml11/DTD/xhtml11.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head id="Head1">
<title>
Quote
</title>
<link href="StyleSheet.css" rel="stylesheet" type="text/css" />
<link rel="StyleSheet" type="text/css" href="/include/arikibo.css" />
<STYLE type="text/css">
td {
font-size: 14px
}
</STYLE>
</head>
<body>
<span STYLE="font-family: Arial, Helvetica, Sans Serif; font-size:20px">
<table cellpadding="3" cellspacing="0" border="0" >
<tr>
<td colspan="5">Please select the type of csv file you wish to generate<br><br>
<b>Please be patient as this may take a few moments!</b><br><br></td>
</tr>
<tr>
<td>Male s</td><td><a href="Name.asp?qt=1&qa=0&ben=1&tpt=0&cl=CAESFFHIILNI&gl=1&life=1&smo=1">Export</a></td>
<td> </td>
<td>Male s</td><td><a href="Name.asp?qt=1&qa=0&ben=1&tpt=0&cl=CAESFFHIILNI&gl=1&life=1&smo=2">Export</a></td>
</tr>
<tr>
<td>Female Non-s</td><td><a href="Name.asp?qt=1&qa=0&ben=1&tpt=0&cl=CAESFFHIILNI&gl=1&life=2&smo=1">Export</a></td>
<td> </td>
<td>Female s</td><td><a href="Name.asp?qt=1&qa=0&ben=1&tpt=0&cl=CAESFFHIILNI&gl=1&life=2&smo=2">Export</a></td>
</tr>
<tr>
<td>Joint Non-s</td><td><a href="Name.asp?qt=1&qa=0&ben=1&tpt=0&cl=CAESFFHIILNI&gl=1&life=3&smo=1">Export</a></td>
<td> </td>
<td>Joint s</td><td><a href="Name.asp?qt=1&qa=0&ben=1&tpt=0&cl=CAESFFHIILNI&gl=1&life=3&smo=2">Export</a></td>
</tr>
<tr>
<td>Dual Non-s</td><td><a href="Name.asp?qt=1&qa=0&ben=1&tpt=0&cl=CAESFFHIILNI&gl=1&life=4&smo=1">Export</a></td>
<td> </td>
<td>Dual s</td><td><a href="Name.asp?qt=1&qa=0&ben=1&tpt=0&cl=CAESFFHIILNI&gl=1&life=4&smo=2">Export</a></td>
</tr>
</table>
</span>
</body>
</html>
答案 0 :(得分：0)
据我了解，弹出窗口是一个新窗口，您必须切换到它：
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
#...
other_url = 'http://somewebsite.com/something/exportToExcelChoice.asp?qt=1&qa=0&ben=1&tpt=0&gl=1&cl=CAESFFHIILNI'
wait = WebDriverWait(browser, 10)
#handles = driver.window_handles
browser.get(other_url)
#wait.until(EC.new_window_is_opened(handles))
#driver.switch_to.window(driver.window_handles[-1])
links = wait.until(EC.visibility_of_all_elements_located((By.TAG_NAME,"a")))
for link in links:
link.click()