Question

我在Scrapy项目中使用Selenium，并且我的代码在我的计算机上可以正常工作。当我尝试在AWS上运行相同版本的代码时，当Scrapy尝试为Firefox设置Selenium Webdriver时，它会说：

2018-11-01 05:27:38 [scrapy.utils.signal] ERROR: Error caught on signal handler: <bound method SeleniumMiddleware.spider_opened of <navigator.middlewares.SeleniumMiddleware object at 0x7ffae89bc470>>
Traceback (most recent call last):
  File "/usr/local/lib/python3.5/dist-packages/twisted/internet/defer.py", line 149, in maybeDeferred
    result = f(*args, **kw)
  File "/usr/local/lib/python3.5/dist-packages/pydispatch/robustapply.py", line 55, in robustApply
    return receiver(*arguments, **named)
  File "/home/ubuntu/search/decapod/navigator/navigator/middlewares.py", line 213, in spider_opened
    self.driver = webdriver.Firefox(executable_path = path, firefox_options=options)
  File "/usr/local/lib/python3.5/dist-packages/selenium/webdriver/firefox/webdriver.py", line 174, in __init__
    keep_alive=True)
  File "/usr/local/lib/python3.5/dist-packages/selenium/webdriver/remote/webdriver.py", line 157, in __init__
    self.start_session(capabilities, browser_profile)
  File "/usr/local/lib/python3.5/dist-packages/selenium/webdriver/remote/webdriver.py", line 252, in start_session
    response = self.execute(Command.NEW_SESSION, parameters)
  File "/usr/local/lib/python3.5/dist-packages/selenium/webdriver/remote/webdriver.py", line 321, in execute
    self.error_handler.check_response(response)
  File "/usr/local/lib/python3.5/dist-packages/selenium/webdriver/remote/errorhandler.py", line 242, in check_response
    raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.WebDriverException: Message: newSession

这是我启动Selenium的代码：

if not self.uses_selenium:
    spider.log("SELENIUM NOT ENABLED", logging.WARNING)
    return
spider.log("SELENIUM ENABLED", logging.WARNING)
path = os.getcwd() + "/geckodriver"
options = webdriver.FirefoxOptions()
if self.headless:
    options.add_argument('--headless')
options.add_argument("--window-size=1600,900")
options.add_argument('--disk-cache=true')
options.add_argument('--load-images=false')
options.add_argument("--user-agent=Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:59.0) Gecko/20100101 Firefox/59.0")
self.driver = webdriver.Firefox(executable_path = path, firefox_options=options)

我想念一些包裹吗？

硒-无法在AWS中创建会话

0 个答案: