Question

我在云计算机（AWS 1GB实例）中运行BOT，我使用的是selenium-webdriver + pyvirtualdisplay。

我在循环中做了一些重复的事情，这将循环50000次。在运行我的脚本后，它在1016（50000）中停止工作。错误是

Traceback (most recent call last):
  File "automation.py", line 108, in <module>
    main()
  File "automation.py", line 10, in main
    driver = webdriver.Firefox()
  File "/home/eric/work/lak/local/lib/python2.7/site-packages/selenium/webdriver/firefox/webdriver.py", line 59, in __init__
    self.binary, timeout),
  File "/home/eric/work/lak/local/lib/python2.7/site-packages/selenium/webdriver/firefox/extension_connection.py", line 47, in __init__
    self.binary.launch_browser(self.profile)
  File "/home/eric/work/lak/local/lib/python2.7/site-packages/selenium/webdriver/firefox/firefox_binary.py", line 61, in launch_browser
    self._wait_until_connectable()
  File "/home/eric/work/lak/local/lib/python2.7/site-packages/selenium/webdriver/firefox/firefox_binary.py", line 105, in _wait_until_connectable
    self.profile.path, self._get_firefox_output()))
selenium.common.exceptions.WebDriverException: Message: 'Can\'t load the profile. Profile Dir: /tmp/tmp7CXXn1 Firefox output: \n(process:22693): GLib-CRITICAL **: g_slice_set_config: assertion \'sys_page_size == 0\' failed\nXlib:  extension "RANDR" missing on display ":1213".\n1406467599310\taddons.manager\tDEBUG\tLoaded provider scope for resource://gre/modules/addons/XPIProvider.jsm: ["XPIProvider"]\n1406467599312\taddons.manager\tDEBUG\tLoaded provider scope for resource://gre/modules/LightweightThemeManager.jsm: ["LightweightThemeManager"]\n1406467599314\taddons.xpi\tDEBUG\tstartup\n1406467599339\taddons.xpi\tDEBUG\tcheckForChanges\n1406467599351\taddons.xpi\tDEBUG\tNo changes found\nSystem JS : ERROR chrome://browser/content/browser.js:12132 - SyntaxError: function statement requires a name\n 
System JS : ERROR (null):0 - out of memory\n
System JS : ERROR (null):0 - out of memory\n
System JS : ERROR (null):0 - out of memory\n
System JS : ERROR (null):0 - out of memory\n
System JS : ERROR (null):0 - out of memory\n
System JS : ERROR (null):0 - out of memory\n
System JS : ERROR (null):0 - out of memory\n'

从错误看来有一些内存问题？是吗？我不知道如何解决它？

事实上如果我重新运行此脚本它不会显示此错误。

selenium.common.exceptions.WebDriverException: Message: 'The browser appears to have exited before we could connect. The output was: \n(process:22882): GLib-CRITICAL **: g_slice_set_config: assertion \'sys_page_size == 0\' failed\nXlib:  extension "RANDR" missing on display ":1295".\n1406470964822\taddons.manager\tDEBUG\tLoaded provider scope for resource://gre/modules/addons/XPIProvider.jsm: ["XPIProvider"]\n1406470964823\taddons.manager\tDEBUG\tLoaded provider scope for resource://gre/modules/LightweightThemeManager.jsm: ["LightweightThemeManager"]\n1406470964826\taddons.xpi\tDEBUG\tstartup\n1406470964847\taddons.xpi\tDEBUG\tcheckForChanges\n1406470964855\taddons.xpi\tDEBUG\tNo changes found\n************************************************************\n* Call to xpconnect wrapped JSObject produced this error:  *\n[Exception... "Component returned failure code: 0x80570016 (NS_ERROR_XPC_GS_RETURNED_FAILURE) [nsIJSCID.getService]"  nsresult: "0x80570016 (NS_ERROR_XPC_GS_RETURNED_FAILURE)"  location: "JS frame :: file:///tmp/tmpdCliUY/extensions/fxdriver@googlecode.com/components/bad_cert_listener.js :: WdCertOverrideService :: line 3855"  data: no]\n************************************************************\n************************************************************\n* Call to xpconnect wrapped JSObject produced this error:  *\n[Exception... "Component returned failure code: 0x80570016 (NS_ERROR_XPC_GS_RETURNED_FAILURE) [nsIJSCID.getService]"  nsresult: "0x80570016 (NS_ERROR_XPC_GS_RETURNED_FAILURE)"  location: "JS frame :: file:///tmp/tmpdCliUY/extensions/fxdriver@googlecode.com/components/bad_cert_listener.js :: WdCertOverrideService :: line 3855"  data: no]\n************************************************************\nJavaScript error: chrome://fxdriver/content/server.js, line 38: NS_ERROR_XPC_CI_RETURNED_FAILURE: Component returned failure code: 0x80570015 (NS_ERROR_XPC_CI_RETURNED_FAILURE) [nsIJSCID.createInstance]\n\n(firefox:22882): LIBDBUSMENU-GLIB-WARNING **: Unable to get session bus: Failed to fork (Cannot allocate memory)\nout of memory: 0x0000000000040028 bytes requested\n'

这里有我的代码，供你参考。

from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from pyvirtualdisplay import Display

display = Display(visible=0, size=(1024, 768))
display.start()
driver = webdriver.Firefox()
driver.maximize_window()

url = 'some url'

for i in range(1,50000):
    driver.get(url)
    #somestuff
    i+=1

Answer 1

从错误日志中看来你正在使用unix flavor

过去我有同样的错误。显然，我们可以看到系统无法提供足够的内存。

运行top命令，查看可用内存是否足够。如果没有，那就杀掉一些不必要的过程。

这应该有用。

Python - 运行几个小时后的Selenium webdriver错误

1 个答案: