用xpath硒查找元素

Question

我正在关注此tutorial，以便学习如何构建网络抓取工具来获取职位列表。现在，我正在尝试跟进另一个网站。我遇到了一个问题，我不知道如何提取各个工作清单的链接。

在检查页面时，我发现了我需要的元素

当我复制xpath并在我的代码中使用它时，出现错误。我究竟做错了什么？

import selenium 
base_url = "https://www.nationalevacaturebank.nl"     
start_url = "https://www.nationalevacaturebank.nl/vacature/zoeken?query=&location=&distance=city&limit=100&sort=relevance&filters%5BcareerLevel%5D%5B%5D=Starter&filters%5BeducationLevel%5D%5B%5D=MBO" 
driver = webdriver.Firefox()    
elem = driver.find_element_by_xpath("//*[@id="search-results-container"]/div/div[1]/div[2]/article/job/a")

>>NoSuchElementException: Message: Unable to locate element: //*[@id="search-results-container"]/div/div[1]/div[2]/article/job/a

Answer 1

首先，您应该连接到您的网站...

第二，您应该使用waits，您可以阅读here

代码应类似于：

from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC 


base_url = "https://www.nationalevacaturebank.nl"     
start_url = "https://www.nationalevacaturebank.nl/vacature/zoeken?query=&location=&distance=city&limit=100&sort=relevance&filters%5BcareerLevel%5D%5B%5D=Starter&filters%5BeducationLevel%5D%5B%5D=MBO" 
my_xpath = '//*[@id="search-results-container"]/div/div[1]/div[2]/article/job/a'
driver = webdriver.Firefox()
driver.get(start_url)  
wait = WebDriverWait(driver, 10)
element = wait.until(EC.element_to_be_clickable((By.XPATH, my_xpath)))

修改

我要获取单个职位清单的所有链接，您可以创建链接列表并附加主题：

wait = WebDriverWait(driver, 10)
elements = wait.until(EC.presence_of_all_elements_located((By.XPATH, '//*[@id="search-results-container"]//article/job/a')))
list_of_links = []
for i in elements:
    list_of_links.append(i.get_attribute('href'))
    # print(f"link = {i.get_attribute('href')}")
print(list_of_links)

希望这对您有帮助！