Question

我正在使用Selenium在Python中运行自动化脚本。我已经使用该工具三年了，但是从未遇到过这个问题。有谁知道是什么原因造成的？我能够确定导致错误的原因是在for循环中对driver.get()的引用，但是经过7次迭代后它出错了。似乎很奇怪，有想法吗？

Unhandled exception in thread started by <function crawl_games_and_store_data.<locals>.handle_incoming_request at 0x104659158>
Traceback (most recent call last):
  File "/Users/z003bzf/Documents/personal/python/MLB/src/services/crawler.py", line 160, in handle_incoming_request
    driver.get(game_link)
  File "/Users/z003bzf/.local/share/virtualenvs/MLB-Ei2Ym8vD/lib/python3.7/site-packages/selenium/webdriver/remote/webdriver.py", line 333, in get
    self.execute(Command.GET, {'url': url})
  File "/Users/z003bzf/.local/share/virtualenvs/MLB-Ei2Ym8vD/lib/python3.7/site-packages/selenium/webdriver/remote/webdriver.py", line 318, in execute
    params = self._wrap_value(params)
  File "/Users/z003bzf/.local/share/virtualenvs/MLB-Ei2Ym8vD/lib/python3.7/site-packages/selenium/webdriver/remote/webdriver.py", line 271, in _wrap_value
    converted[key] = self._wrap_value(val)
  File "/Users/z003bzf/.local/share/virtualenvs/MLB-Ei2Ym8vD/lib/python3.7/site-packages/selenium/webdriver/remote/webdriver.py", line 276, in _wrap_value
    return list(self._wrap_value(item) for item in value)

这是引起问题的代码

for elem in link_lst:
     driver.get(elem)
     time.sleep(.5)

     driver.find_element_by_xpath('//div[@class="box-row batting-row"]')

Answer 1

如果其中一台主机超时，则可能是link_lst的内容。您必须处理此异常才能继续前进。一种可能的选择是对超时使用try / except以及无法找到页面元素。既可以将其作为延迟参数进行调整，也可以在firefox配置文件设置中进行调整。

from selenium import webdriver
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.by import By
from selenium.common.exceptions import TimeoutException


def fetch(driver, link_list, delay)
    for item in link_list:
        try:
            driver.get(item)
        except TimeoutException:
            print("Timeout in link_list item")

        try:
            elem = WebDriverWait(driver, 
            delay).until(EC.presence_of_element_located((By.XPATH '//div[@class="box-row batting-row"]')))
        except TimeoutException:
            print("Locating element took too much time!")

if __name__ == '__main__':

    """ Setup profile """
    fp = webdriver.FirefoxProfile()
    fp.set_preference("http.response.timeout", 10)
    fp.set_preference("dom.max_script_run_time", 10)

    """ Prepare args """
    driver = webdriver.Firefox(firefox_profile=fp)
    link_list = ['http://abcdeftest.com', 'http://test.com']
    delay = 5

    fetch(driver, link_list, delay)

Selenium的driver.get（）调用的最大递归错误

1 个答案: