硒测试在通过10次测试后在CircleCI中失败

时间:2019-08-23 17:00:58

标签: python selenium selenium-webdriver circleci zalenium

在测试套件增加之后,硒测试开始失败。通常,在达到10个通过的测试之后,测试将失败,此后,所有其余测试都将失败,并且没有触发重试的信息。所有这些测试都在本地成功通过,我尝试将它们混合并创建不同的订单批次,但是,CI中仍然通过了前十项测试,其余测试均失败了。 这些失败并非每次代码推送都发生,而是每3次代码推送发生一次。

尝试混合测试,在本地多次运行以确保没有失败。由于dockerized环境中的内存问题,我更改了webdriver实例的范围,现在它在新环境中运行每个测试,而以前运行的测试中没有数据。

这些故障刚开始出现时,我在日志中还没有看到,只有10个通过测试和消息console log output exceeded timeout,所以我向circleci/config.yml添加了一个参数 no_output_timeout: 20m 开始显示其余测试失败。

platform linux -- Python 3.6.4, pytest-4.3.1, py-1.8.0, pluggy-0.9.0 -- /usr/local/bin/python3
cachedir: .pytest_cache
metadata: {'Python': '3.6.4', 'Platform': 'Linux-4.4.0-96-generic-x86_64-with', 'Packages': {'pytest': '4.3.1', 'py': '1.8.0', 'pluggy': '0.9.0'}, 'Plugins': {'reportportal': '1.0.4', 'parallel': '0.0.9', 'metadata': '1.8.0', 'html': '1.20.0', 'flaky': '3.5.3'}}
rootdir: /home/robot/functional/retail, inifile:
plugins: reportportal-1.0.4, parallel-0.0.9, metadata-1.8.0, html-1.20.0, flaky-3.5.3
collecting ... 
collected 36 items                                                             

tests/ui_workflows/test_ecom_publish.py::TestEcomPublish::test_create_new_item_and_publish_on_ecom[firefox] SKIPPED
tests/ui_workflows/test_retail_create.py::TestRetailCreate::test_create_new_item[firefox] Using Selenium grid at http://selenium-grid:4444/wd/hub
PASSED
tests/ui_workflows/test_retail_create.py::TestRetailCreate::test_create_new_customer_type[firefox] Using Selenium grid at http://selenium-grid:4444/wd/hub
PASSED
tests/ui_workflows/test_retail_create.py::TestRetailCreate::test_create_new_sales_tax[firefox] Using Selenium grid at http://selenium-grid:4444/wd/hub
PASSED
tests/ui_workflows/test_retail_import.py::TestRetailCreate::test_import_items_from_csv[firefox] Using Selenium grid at http://selenium-grid:4444/wd/hub
PASSED
tests/ui_workflows/test_retail_import.py::TestRetailCreate::test_import_items_from_xlsx[firefox] Using Selenium grid at http://selenium-grid:4444/wd/hub
PASSED
tests/ui_workflows/test_retail_import.py::TestRetailCreate::test_import_300_items_from_xlsx[firefox] Using Selenium grid at http://selenium-grid:4444/wd/hub
PASSED
tests/ui_workflows/test_retail_item.py::TestRetailItem::test_create_item_with_all_attributes[firefox] Using Selenium grid at http://selenium-grid:4444/wd/hub
PASSED
tests/ui_workflows/test_retail_item.py::TestRetailItem::test_create_non_taxable_item_and_make_sale[firefox] Using Selenium grid at http://selenium-grid:4444/wd/hub
PASSED
tests/ui_workflows/test_retail_item.py::TestRetailItem::test_create_non_discountable_item_and_make_sale[firefox] Using Selenium grid at http://selenium-grid:4444/wd/hub
PASSED
tests/ui_workflows/test_retail_item.py::TestRetailItem::test_create_non_discount_non_tax_item_and_make_sale[firefox] Using Selenium grid at http://selenium-grid:4444/wd/hub
PASSED
tests/ui_workflows/test_retail_item.py::TestRetailItem::test_create_item_and_quick_edit_all_attr[firefox] Using Selenium grid at http://selenium-grid:4444/wd/hub
ERROR
tests/ui_workflows/test_retail_item_matrix.py::TestRetailMatrix::test_create_color_size_attributes_matrix[firefox] Using Selenium grid at http://selenium-grid:4444/wd/hub
ERROR
tests/ui_workflows/test_retail_item_matrix.py::TestRetailMatrix::test_add_4th_attr_color_size_attr_matrix[firefox] Using Selenium grid at http://selenium-grid:4444/wd/hub
ERROR
tests/ui_workflows/test_retail_item_matrix.py::TestRetailMatrix::test_create_color_attributes_matrix[firefox] Using Selenium grid at http://selenium-grid:4444/wd/hub
ERROR
tests/ui_workflows/test_retail_item_matrix.py::TestRetailMatrix::test_add_4th_color_attribute_matrix[firefox] Using Selenium grid at http://selenium-grid:4444/wd/hub
ERROR
tests/ui_workflows/test_retail_item_matrix.py::TestRetailMatrix::test_create_size_attributes_matrix[firefox] Using Selenium grid at http://selenium-grid:4444/wd/hub
ERROR
==================================== ERRORS ====================================
_ ERROR at setup of TestRetailItem.test_create_item_and_quick_edit_all_attr[firefox] _

>       lambda: ihook(item=item, **kwds),
        when=when,
    )

/usr/local/lib/python3.6/site-packages/flaky/flaky_pytest_plugin.py:306: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
tests/ui_workflows/conftest.py:89: in driver
    driver = webdriver.Remote(command_executor=selenium_grid_endpoint, desired_capabilities=capabilities)
/usr/local/lib/python3.6/site-packages/selenium/webdriver/remote/webdriver.py:156: in __init__
    self.start_session(capabilities, browser_profile)
/usr/local/lib/python3.6/site-packages/selenium/webdriver/remote/webdriver.py:251: in start_session
    response = self.execute(Command.NEW_SESSION, parameters)
/usr/local/lib/python3.6/site-packages/selenium/webdriver/remote/webdriver.py:320: in execute
    self.error_handler.check_response(response)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <selenium.webdriver.remote.errorhandler.ErrorHandler object at 0x7f30b61d3d68>
response = {'status': 500, 'value': '{\n  "sessionId": null,\n  "status": 13,\n  "value": {\n    "class": "org.openqa.grid.common...\tat org.openqa.grid.web.servlet.handler.RequestHandler.process(RequestHandler.java:112)\\n\\t... 38 more\\n"\n  }\n}'}

    def check_response(self, response):
        """
        Checks that a JSON response from the WebDriver does not have an error.

        :Args:
         - response - The JSON response from the WebDriver server as a dictionary
           object.

        :Raises: If the response contains an error message.
        """
        status = response.get('status', None)
        if status is None or status == ErrorCode.SUCCESS:
            return
        value = None
        message = response.get("message", "")
        screen = response.get("screen", "")
        stacktrace = None
        if isinstance(status, int):
            value_json = response.get('value', None)
            if value_json and isinstance(value_json, basestring):
                import json
                try:
                    value = json.loads(value_json)
                    if len(value.keys()) == 1:
                        value = value['value']
                    status = value.get('error', None)
                    if status is None:
                        status = value["status"]
                        message = value["value"]
                        if not isinstance(message, basestring):
                            value = message
                            message = message.get('message')
                    else:
                        message = value.get('message', None)
                except ValueError:
                    pass

        exception_class = ErrorInResponseException
        if status in ErrorCode.NO_SUCH_ELEMENT:
            exception_class = NoSuchElementException
        elif status in ErrorCode.NO_SUCH_FRAME:
            exception_class = NoSuchFrameException
        elif status in ErrorCode.NO_SUCH_WINDOW:
            exception_class = NoSuchWindowException
        elif status in ErrorCode.STALE_ELEMENT_REFERENCE:
            exception_class = StaleElementReferenceException
        elif status in ErrorCode.ELEMENT_NOT_VISIBLE:
            exception_class = ElementNotVisibleException
        elif status in ErrorCode.INVALID_ELEMENT_STATE:
            exception_class = InvalidElementStateException
        elif status in ErrorCode.INVALID_SELECTOR \
                or status in ErrorCode.INVALID_XPATH_SELECTOR \
                or status in ErrorCode.INVALID_XPATH_SELECTOR_RETURN_TYPER:
            exception_class = InvalidSelectorException
        elif status in ErrorCode.ELEMENT_IS_NOT_SELECTABLE:
            exception_class = ElementNotSelectableException
        elif status in ErrorCode.ELEMENT_NOT_INTERACTABLE:
            exception_class = ElementNotInteractableException
        elif status in ErrorCode.INVALID_COOKIE_DOMAIN:
            exception_class = InvalidCookieDomainException
        elif status in ErrorCode.UNABLE_TO_SET_COOKIE:
            exception_class = UnableToSetCookieException
        elif status in ErrorCode.TIMEOUT:
            exception_class = TimeoutException
        elif status in ErrorCode.SCRIPT_TIMEOUT:
            exception_class = TimeoutException
        elif status in ErrorCode.UNKNOWN_ERROR:
            exception_class = WebDriverException
        elif status in ErrorCode.UNEXPECTED_ALERT_OPEN:
            exception_class = UnexpectedAlertPresentException
        elif status in ErrorCode.NO_ALERT_OPEN:
            exception_class = NoAlertPresentException
        elif status in ErrorCode.IME_NOT_AVAILABLE:
            exception_class = ImeNotAvailableException
        elif status in ErrorCode.IME_ENGINE_ACTIVATION_FAILED:
            exception_class = ImeActivationFailedException
        elif status in ErrorCode.MOVE_TARGET_OUT_OF_BOUNDS:
            exception_class = MoveTargetOutOfBoundsException
        elif status in ErrorCode.JAVASCRIPT_ERROR:
            exception_class = JavascriptException
        elif status in ErrorCode.SESSION_NOT_CREATED:
            exception_class = SessionNotCreatedException
        elif status in ErrorCode.INVALID_ARGUMENT:
            exception_class = InvalidArgumentException
        elif status in ErrorCode.NO_SUCH_COOKIE:
            exception_class = NoSuchCookieException
        elif status in ErrorCode.UNABLE_TO_CAPTURE_SCREEN:
            exception_class = ScreenshotException
        elif status in ErrorCode.ELEMENT_CLICK_INTERCEPTED:
            exception_class = ElementClickInterceptedException
        elif status in ErrorCode.INSECURE_CERTIFICATE:
            exception_class = InsecureCertificateException
        elif status in ErrorCode.INVALID_COORDINATES:
            exception_class = InvalidCoordinatesException
        elif status in ErrorCode.INVALID_SESSION_ID:
            exception_class = InvalidSessionIdException
        elif status in ErrorCode.UNKNOWN_METHOD:
            exception_class = UnknownMethodException
        else:
            exception_class = WebDriverException
        if value == '' or value is None:
            value = response['value']
        if isinstance(value, basestring):
            if exception_class == ErrorInResponseException:
                raise exception_class(response, value)
            raise exception_class(value)
        if message == "" and 'message' in value:
            message = value['message']

        screen = None
        if 'screen' in value:
            screen = value['screen']

        stacktrace = None
        if 'stackTrace' in value and value['stackTrace']:
            stacktrace = []
            try:
                for frame in value['stackTrace']:
                    line = self._value_or_default(frame, 'lineNumber', '')
                    file = self._value_or_default(frame, 'fileName', '<anonymous>')
                    if line:
                        file = "%s:%s" % (file, line)
                    meth = self._value_or_default(frame, 'methodName', '<anonymous>')
                    if 'className' in frame:
                        meth = "%s.%s" % (frame['className'], meth)
                    msg = "    at %s (%s)"
                    msg = msg % (meth, file)
                    stacktrace.append(msg)
            except TypeError:
                pass
        if exception_class == ErrorInResponseException:
            raise exception_class(response, message)
        elif exception_class == UnexpectedAlertPresentException:
            alert_text = None
            if 'data' in value:
                alert_text = value['data'].get('text')
            elif 'alert' in value:
                alert_text = value['alert'].get('text')
            raise exception_class(message, screen, stacktrace, alert_text)
>       raise exception_class(message, screen, stacktrace)
E       selenium.common.exceptions.WebDriverException: Message: Error forwarding the new session Request timed out waiting for a node to become available.
E       Stacktrace:
E           at org.openqa.grid.web.servlet.handler.RequestHandler.process (RequestHandler.java:118)
E           at org.openqa.grid.web.servlet.DriverServlet.process (DriverServlet.java:85)
E           at org.openqa.grid.web.servlet.DriverServlet.doPost (DriverServlet.java:69)
E           at javax.servlet.http.HttpServlet.service (HttpServlet.java:707)
E           at javax.servlet.http.HttpServlet.service (HttpServlet.java:790)
E           at org.seleniumhq.jetty9.servlet.ServletHolder.handle (ServletHolder.java:865)
E           at org.seleniumhq.jetty9.servlet.ServletHandler$CachedChain.doFilter (ServletHandler.java:1655)
E           at io.prometheus.client.filter.MetricsFilter.doFilter (MetricsFilter.java:170)
E           at org.seleniumhq.jetty9.servlet.ServletHandler$CachedChain.doFilter (ServletHandler.java:1642)
E           at org.seleniumhq.jetty9.servlet.ServletHandler.doHandle (ServletHandler.java:533)
E           at org.seleniumhq.jetty9.server.handler.ScopedHandler.handle (ScopedHandler.java:146)
E           at org.seleniumhq.jetty9.security.SecurityHandler.handle (SecurityHandler.java:548)
E           at org.seleniumhq.jetty9.server.handler.HandlerWrapper.handle (HandlerWrapper.java:132)
E           at org.seleniumhq.jetty9.server.handler.ScopedHandler.nextHandle (ScopedHandler.java:257)
E           at org.seleniumhq.jetty9.server.session.SessionHandler.doHandle (SessionHandler.java:1595)
E           at org.seleniumhq.jetty9.server.handler.ScopedHandler.nextHandle (ScopedHandler.java:255)
E           at org.seleniumhq.jetty9.server.handler.ContextHandler.doHandle (ContextHandler.java:1340)
E           at org.seleniumhq.jetty9.server.handler.ScopedHandler.nextScope (ScopedHandler.java:203)
E           at org.seleniumhq.jetty9.servlet.ServletHandler.doScope (ServletHandler.java:473)
E           at org.seleniumhq.jetty9.server.session.SessionHandler.doScope (SessionHandler.java:1564)
E           at org.seleniumhq.jetty9.server.handler.ScopedHandler.nextScope (ScopedHandler.java:201)
E           at org.seleniumhq.jetty9.server.handler.ContextHandler.doScope (ContextHandler.java:1242)
E           at org.seleniumhq.jetty9.server.handler.ScopedHandler.handle (ScopedHandler.java:144)
E           at org.seleniumhq.jetty9.server.handler.StatisticsHandler.handle (StatisticsHandler.java:174)
E           at org.seleniumhq.jetty9.server.handler.HandlerWrapper.handle (HandlerWrapper.java:132)
E           at org.seleniumhq.jetty9.server.Server.handle (Server.java:503)
E           at org.seleniumhq.jetty9.server.HttpChannel.handle (HttpChannel.java:364)
E           at org.seleniumhq.jetty9.server.HttpConnection.onFillable (HttpConnection.java:260)
E           at org.seleniumhq.jetty9.io.AbstractConnection$ReadCallback.succeeded (AbstractConnection.java:305)
E           at org.seleniumhq.jetty9.io.FillInterest.fillable (FillInterest.java:103)
E           at org.seleniumhq.jetty9.io.ChannelEndPoint$2.run (ChannelEndPoint.java:118)
E           at org.seleniumhq.jetty9.util.thread.strategy.EatWhatYouKill.runTask (EatWhatYouKill.java:333)
E           at org.seleniumhq.jetty9.util.thread.strategy.EatWhatYouKill.doProduce (EatWhatYouKill.java:310)
E           at org.seleniumhq.jetty9.util.thread.strategy.EatWhatYouKill.tryProduce (EatWhatYouKill.java:168)
E           at org.seleniumhq.jetty9.util.thread.strategy.EatWhatYouKill.run (EatWhatYouKill.java:126)
E           at org.seleniumhq.jetty9.util.thread.ReservedThreadExecutor$ReservedThread.run (ReservedThreadExecutor.java:366)
E           at org.seleniumhq.jetty9.util.thread.QueuedThreadPool.runJob (QueuedThreadPool.java:765)
E           at org.seleniumhq.jetty9.util.thread.QueuedThreadPool$2.run (QueuedThreadPool.java:683)
E           at java.lang.Thread.run (Thread.java:748)

/usr/local/lib/python3.6/site-packages/selenium/webdriver/remote/errorhandler.py:242: WebDriverException

0 个答案:

没有答案