Question

我正在使用pytest.mark.parametrize将越来越长的输入输入到相当慢的测试函数中，例如：

@pytest.mark.parametrize('data', [
    b'ab',
    b'xyz'*1000,
    b'12345'*1024**2,
    ... # etc
])
def test_compression(data):
    ... # compress the data
    ... # decompress the data
    assert decompressed_data == data

由于压缩大量数据需要很长时间，因此我想在一次失败后跳过所有剩余的测试。例如，如果测试失败并输入b'ab'（第一个），b'xyz'*1000和b'12345'*1024**2，则应跳过所有其他参数设置（或不执行xfail即可）。

我知道可以将标记附加到各个参数，例如：

@pytest.mark.parametrize("test_input,expected", [
    ("3+5", 8),
    ("2+4", 6),
    pytest.param("6*9", 42, marks=pytest.mark.xfail),
])

但是我不知道如何根据先前测试用例的状态有条件应用这些标记。有办法吗？

Answer 1

在执行测试之前会对标记进行评估，因此无法传递依赖于其他测试结果的某种声明性标记（如skipif）。不过，您可以在挂钩中应用自定义测试跳过逻辑。从pytest文档中修改incremental testing - test steps食谱：

# conftest.py
import pytest

def pytest_sessionstart(session):
    session.failednames = set()

def pytest_runtest_makereport(item, call):
    if call.excinfo is not None:
        item.session.failednames.add(item.originalname)

def pytest_runtest_setup(item):
    if item.originalname in item.session.failednames:
        pytest.skip("previous test failed (%s)" % item.name)  # or use pytest.xfail like in the other answer

示例测试

@pytest.mark.parametrize('i', range(10))
def test_spam(i):
    assert i != 3

产量：

=================================== test session starts ===================================
collected 10 items

test_spam.py::test_spam[0] PASSED
test_spam.py::test_spam[1] PASSED
test_spam.py::test_spam[2] PASSED
test_spam.py::test_spam[3] FAILED
test_spam.py::test_spam[4] SKIPPED
test_spam.py::test_spam[5] SKIPPED
test_spam.py::test_spam[6] SKIPPED
test_spam.py::test_spam[7] SKIPPED
test_spam.py::test_spam[8] SKIPPED
test_spam.py::test_spam[9] SKIPPED

========================================= FAILURES ========================================
_______________________________________ test_spam[3] ______________________________________

i = 3

    @pytest.mark.parametrize('i', range(10))
    def test_spam(i):
>       assert i != 3
E       assert 3 != 3

test_spam.py:5: AssertionError
====================== 1 failed, 3 passed, 6 skipped in 0.06 seconds ======================

编辑：使用自定义标记

def pytest_runtest_makereport(item, call):
    markers = {marker.name for marker in item.iter_markers()}
    if call.excinfo is not None and 'skiprest' in markers:
        item.session.failednames.add(item.originalname)

def pytest_runtest_setup(item):
    markers = {marker.name for marker in item.iter_markers()}
    if item.originalname in item.session.failednames and 'skiprest' in markers:
        pytest.skip(item.name)

用法：

@pytest.mark.skiprest
@pytest.mark.parametrize('somearg', ['a', 'b', 'c'])
def test_marked(somearg):
    ...

Answer 2

接受的答案不再有效，item.session 没有 failednames 参数。

灵感来自https://docs.pytest.org/en/latest/example/simple.html#incremental-testing-test-steps

还有一种与公认的非常接近的解决方案：

conftest.py

MARKER_SKIPREST = "skiprest"
MARKER_PARAMETRIZE = "parametrize"

_failed_parametrized_tests = set()  # History of failed tests

def pytest_runtest_makereport(item, call):
    """Memorizes names of failed Parametrized tests"""
    marker_names = {marker.name for marker in item.iter_markers()}
    if {MARKER_SKIPREST, MARKER_PARAMETRIZE}.issubset(marker_names):
        if call.excinfo is not None:
            _failed_parametrized_tests.add(item.originalname)


def pytest_runtest_setup(item):
    """Check if the test has already failed with other param.
    If yes - xfail this test"""
    marker_names = {marker.name for marker in item.iter_markers()}
    if {MARKER_SKIPREST, MARKER_PARAMETRIZE}.issubset(marker_names):
        if item.originalname in _failed_parametrized_tests:
            pytest.xfail("Previous test failed")

setup.cfg

[tool:pytest]
markers =
    skiprest: Skip rest of params in parametrized test if one test one of the tests in the sequence has failed.

test_?.py

pytest.mark.skiprest
pytest.mark.parametrize('data', [
    b'ab',
    b'xyz'*1000,
    b'12345'*1024**2,
    ... # etc
])
def test_compression(data):
   ...

使用pytest的参数设置，如果一个测试用例失败，如何跳过其余测试？

2 个答案:

编辑：使用自定义标记