Question

test.py

#!/usr/bin/env python2
import os
from pyvirtualdisplay import Display
from selenium import webdriver
import sys

display = Display(visible=0, size=(800, 600))
display.start()
try:
    capabilities = webdriver.common.desired_capabilities.DesiredCapabilities.CHROME
    browser = webdriver.Chrome('/opt/chrome-driver/chromedriver', desired_capabilities=capabilities)
    try:
        browser.get('http://example.com')
        browser.execute_script('return 1 + 1;')
    finally:
        browser.quit()
finally:
    display.stop()
print 'Done'

然后运行

seq 1 20 | parallel -j 5 ./test.py

我得到一个随机数的错误：

Traceback (most recent call last):
  File "./test.py", line 15, in <module>
    browser.execute_script('return 1 + 1;')
  File "/usr/local/lib/python2.7/dist-packages/selenium/webdriver/remote/webdriver.py", line 429, in execute_script
    {'script': script, 'args':converted_args})['value']
  File "/usr/local/lib/python2.7/dist-packages/selenium/webdriver/remote/webdriver.py", line 201, in execute
    self.error_handler.check_response(response)
  File "/usr/local/lib/python2.7/dist-packages/selenium/webdriver/remote/errorhandler.py", line 194, in check_response
    raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.WebDriverException: Message: chrome not reachable
  (Session info: chrome=50.0.2661.86)
  (Driver info: chromedriver=2.21.371461 (633e689b520b25f3e264a2ede6b74ccc23cb636a),platform=Linux 3.19.0-59-generic x86_64)

如果删除并发（parallel -j 1），它就可以工作。

如果我删除Xvfb，它就可以。

这个错误是什么意思，我该如何修复它（不放弃并发或虚拟显示）？

Answer 1

在启动时看起来启动xvfb会出现并发问题。 xvfb-run unreliable when multiple instances invoked in parallel

所以我可以通过要求xfvb会话连续启动来解决这个问题。

#!/usr/bin/env python2
import fcntl
import os
from pyvirtualdisplay import Display
from selenium import webdriver
import sys
import xvfbwrapper

display = None
browser = None
try:
    with open('/tmp/xvfb.lock', 'w') as lock:
        fcntl.lockf(lock, fcntl.LOCK_EX)
        display = Display(visible=0, size=(800, 600))
        display.start()
        capabilities = webdriver.common.desired_capabilities.DesiredCapabilities.CHROME
        browser = webdriver.Chrome('/opt/chrome-driver/chromedriver', desired_capabilities=capabilities)
    browser.get('http://example.com')
    browser.execute_script('return 1 + 1;')
finally:
    if browser:
        browser.quit()
    if display:
        display.stop()
print 'Done'

只要每个人都使用/tmp/xvfb.lock来控制启动Xvfb会话，就可以了。

P.S。我不认为硒驱动器的启动也需要序列化，但我确实遇到了一个短暂的问题，我认为这是由此解决的。

Selenium + Xvfb并发问题

1 个答案: