Question

我使用Selenium Webdriver在一个网站上进行搜索，该网站为我提供了多个页面，每个页面上都有多个链接。我的目标是创建搜索产生的所有链接的列表。

对于第一页它工作正常，但是当我在第二页上时，我尝试提取“ href”属性的for循环会产生StaleElementReferenceException。

我相信应该有某种方法可以通过WebDriverWait来解决此问题，但是我不太清楚该怎么做。感谢您的任何建议。

links =[]

while True:

  result = driver.find_element_by_id('results')
# list of all relevant elements on the page
  docs = result.find_elements_by_xpath('//li[@class="row1"]')

  for d in docs:
# this line produces StaleElementReferenceException
    link = d.find_element_by_tag_name('a')
    links.append(link.get_attribute('href'))

# try block checks if "next page" button exists and clicks it (this works fine)
  try:
    next_page = WebDriverWait(driver, 30).until(EC.presence_of_element_located((By.XPATH, '//a[contains(@class,"la-TriangleRight")]')))
    driver.execute_script('arguments[0].click();', next_page)

# if "next page" button doesn't exist we break out of the loop
  except:
    break

编辑：错误消息：

StaleElementReferenceException            Traceback (most recent call last)
<ipython-input-6-ad6166a696e6> in <module>
     23 #         link = WebDriverWait(d, 30).until(EC.presence_of_element_located((By.XPATH, '//a')))
     24         link = d.find_element_by_tag_name('a')
---> 25         links.append(link.get_attribute('href'))
     26 
     27 #     for i in range(25):

/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/selenium/webdriver/remote/webelement.py in get_attribute(self, name)
    139             attributeValue = self.parent.execute_script(
    140                 "return (%s).apply(null, arguments);" % getAttribute_js,
--> 141                 self, name)
    142         else:
    143             resp = self._execute(Command.GET_ELEMENT_ATTRIBUTE, {'name': name})

/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/selenium/webdriver/remote/webdriver.py in execute_script(self, script, *args)
    634         return self.execute(command, {
    635             'script': script,
--> 636             'args': converted_args})['value']
    637 
    638     def execute_async_script(self, script, *args):

/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/selenium/webdriver/remote/webdriver.py in execute(self, driver_command, params)
    319         response = self.command_executor.execute(driver_command, params)
    320         if response:
--> 321             self.error_handler.check_response(response)
    322             response['value'] = self._unwrap_value(
    323                 response.get('value', None))

/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/selenium/webdriver/remote/errorhandler.py in check_response(self, response)
    240                 alert_text = value['alert'].get('text')
    241             raise exception_class(message, screen, stacktrace, alert_text)
--> 242         raise exception_class(message, screen, stacktrace)
    243 
    244     def _value_or_default(self, obj, key, default):

StaleElementReferenceException: Message: stale element reference: element is not attached to the page document
  (Session info: chrome=75.0.3770.100)

Answer 1

StaleElementReferenceException出现在您要操作的元素不再存在于html中或变得陈旧（通常发生在对页面上的元素进行操作后刷新页面时）。因此，要更正它，您必须再次获取元素。

根据您的情况，您可以像这样纠正它：

links =[]

while True:

  result = driver.find_element_by_id('results')
  docs = result.find_elements_by_xpath('//li[@class="row1"]')

  for d in docs:
    link = d.find_element_by_tag_name('a')
    links.append(link.get_attribute('href'))

  try:
    next_page = WebDriverWait(driver, 30).until(EC.presence_of_element_located((By.XPATH, '//a[contains(@class,"la-TriangleRight")]')))
    driver.execute_script('arguments[0].click();', next_page)
    # Fetch the elements again here
    result = driver.find_element_by_id('results')
    docs = result.find_elements_by_xpath('//li[@class="row1"]')

  except:
    break

Answer 2

对于每个循环，请尝试使用带范围的for循环。

links =[]

while True:

  result = driver.find_element_by_id('results')
# list of all relevant elements on the page
  docs = result.find_elements_by_xpath('//li[@class="row1"]')

  for i in range(len(docs)):
# this line produces StaleElementReferenceException
    WebDriverWait(driver, 30).until(EC.staleness_of(docs[i]))
    link = docs[i].find_element_by_tag_name('a')
    links.append(link.get_attribute('href'))
    result = driver.find_element_by_id('results')
    docs = result.find_elements_by_xpath('//li[@class="row1"]')

# try block checks if "next page" button exists and clicks it (this works fine)
  try:
    next_page = WebDriverWait(driver, 30).until(EC.presence_of_element_located((By.XPATH, '//a[contains(@class,"la-TriangleRight")]')))
    driver.execute_script('arguments[0].click();', next_page)

# if "next page" button doesn't exist we break out of the loop
  except:
    break

使用硒（python）遍历Web元素列表以检索属性会产生StaleElementReferenceException

2 个答案: