Question

我正在尝试通过网络抓取一个网站来获取有关足球比赛的信息。因此，我在Python中使用Selenium库。

我将来自所有必需匹配项的可单击html元素存储在名为“ completed_matches”的列表中。我创建了一个for循环，该循环遍历所有这些可单击的html元素。在循环中，我单击当前的html元素并打印新的URL。代码如下：

from selenium import webdriver
import selenium
from selenium.webdriver.support.ui import WebDriverWait

driver = webdriver.Chrome(r"C:\Users\Mart\Downloads\chromedriver_win32_2\chromedriver.exe")
url = "https://footystats.org/spain/la-liga/matches"
driver.get(url)
completed_matches = driver.find_elements_by_xpath("""//*[@id="matches-list"]/div[@class='full-matches-table mt2e ' or @class='full-matches-table mt1e ']/div/div[2]/table[@class='matches-table inactive-matches']/tbody/tr[*]/td[3]/a[1]/span""");
print(len(completed_matches))
for match in completed_matches:
        match.click()
        print("Current driver URL: " + driver.current_url)

输出看起来像这样：

159
Current driver URL: https://footystats.org/spain/fc-barcelona-vs-real-club-deportivo-mallorca-h2h-stats#632514
---------------------------------------------------------------------------
StaleElementReferenceException            Traceback (most recent call last)
<ipython-input-3-da5851d767a8> in <module>
      4 print(len(completed_matches))
      5 for match in completed_matches:
----> 6         match.click()
      7         print("Current driver URL: " + driver.current_url)

~\Anaconda3\lib\site-packages\selenium\webdriver\remote\webelement.py in click(self)
     78     def click(self):
     79         """Clicks the element."""
---> 80         self._execute(Command.CLICK_ELEMENT)
     81 
     82     def submit(self):

~\Anaconda3\lib\site-packages\selenium\webdriver\remote\webelement.py in _execute(self, command, params)
    631             params = {}
    632         params['id'] = self._id
--> 633         return self._parent.execute(command, params)
    634 
    635     def find_element(self, by=By.ID, value=None):

~\Anaconda3\lib\site-packages\selenium\webdriver\remote\webdriver.py in execute(self, driver_command, params)
    319         response = self.command_executor.execute(driver_command, params)
    320         if response:
--> 321             self.error_handler.check_response(response)
    322             response['value'] = self._unwrap_value(
    323                 response.get('value', None))

~\Anaconda3\lib\site-packages\selenium\webdriver\remote\errorhandler.py in check_response(self, response)
    240                 alert_text = value['alert'].get('text')
    241             raise exception_class(message, screen, stacktrace, alert_text)
--> 242         raise exception_class(message, screen, stacktrace)
    243 
    244     def _value_or_default(self, obj, key, default):

StaleElementReferenceException: Message: stale element reference: element is not attached to the page document
  (Session info: chrome=79.0.3945.79)
  (Driver info: chromedriver=72.0.3626.7 (efcef9a3ecda02b2132af215116a03852d08b9cb),platform=Windows NT 10.0.18362 x86_64)

completed_matches列表包含159个html元素，但是for循环仅显示第一个单击的链接，然后引发StaleElementReferenceException ...

有人知道如何解决这个问题吗？

Answer 1

您要查找的URL在您单击的链接中。您要选择单击的父元素。 StaleElementReferenceException是因为单击链接后页面会更改，以呈现第一个单击过时的元素后的所有元素。

from selenium import webdriver
import selenium
from selenium.webdriver.support.ui import WebDriverWait

driver = webdriver.Chrome(r"C:\Users\Mart\Downloads\chromedriver_win32_2\chromedriver.exe")
url = "https://footystats.org/spain/la-liga/matches"
driver.get(url)
completed_matches = driver.find_elements_by_xpath("""//*[@id="matches-list"]/div[@class='full-matches-table mt2e ' or @class='full-matches-table mt1e ']/div/div[2]/table[@class='matches-table inactive-matches']/tbody/tr[*]/td[3]/a[1]/span""");
print(len(completed_matches))
for match in completed_matches:
        #match.click()
        #print("Current driver URL: " + driver.current_url)
        match_parent = match.find_element_by_xpath("..")
        href = match_parent.get_attribute("href")
        print("href: ", href)

Answer 2

单击后，DOM会刷新，因此StaleElementReferenceException。因此，在for循环内再次构建 completed_matches 元素。

completed_matches = driver.find_elements_by_xpath("""//*[@id="matches-list"]/div[@class='full-matches-table mt2e ' or @class='full-matches-table mt1e ']/div/div[2]/table[@class='matches-table inactive-matches']/tbody/tr[*]/td[3]/a[1]/span""");
print(len(completed_matches))
for match in completed_matches:
    completed_matches = driver.find_elements_by_xpath("""//*[@id="matches-list"]/div[@class='full-matches-table mt2e ' or @class='full-matches-table mt1e ']/div/div[2]/table[@class='matches-table inactive-matches']/tbody/tr[*]/td[3]/a[1]/span""");
    match.click()

Answer 3

陈旧意味着旧的，腐烂的，不再新鲜的。陈旧元素表示旧元素或不再可用的元素。假设在WebDriver中引用为WebElement的网页上找到一个元素。如果DOM更改，则WebElement会过时。

所以这意味着单击该元素后您正在工作的页面正在更改，所以我建议您修复此问题：

from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
while True:
     try:
         completed_match = WebDriverWait(driver,10).until(EC.element_to_be_clickable((By.XPATH, "//*[@id="matches-list"]/div[@class='full-matches-table mt2e ")))
     except TimeoutException:
           break
     completed_match.click()
     time.sleep(2)

因此，只需遍历元素并每次进行更新，在这种情况下，它肯定会出现在页面的DOM中

您可以在此处查看带有详细信息代码的Trip Advisor的网页抓取器：

https://github.com/alirezaznz/Tripadvisor-Webscraper

循环槽元素（硒）时如何解决StaleElementReferenceException

3 个答案: