Question

我向DB写了一些插件-可能会改变从DB接收到的结果，但是大多数情况下是不希望的。我想知道它何时发生。

我有几十个测试，并且我为任何功能添加了更多测试，并且我希望拥有一个系统，其中所有测试一次运行一次，无需该插件即可使数据库运行，然后使用该插件并可以比较结果。我需要它准备好进行更多测试。

目前，无论数据库是否支持该插件，我都可以更改夹具。每次使用不同的夹具运行时，是否可以选择使测试运行两次？

Answer 1

除非我误解了您的问题，否则您可以定义参数化的灯具，该灯具将基于当前参数（真实或模拟）选择特定的展示。这是一个将sqlalchemy与SQLite数据库和alchemy-mock结合使用的工作示例：

import pytest
from unittest import mock

from sqlalchemy import create_engine, Column, String, Integer
from sqlalchemy.ext.declarative import declarative_base
from sqlalchemy.orm import sessionmaker

from alchemy_mock.mocking import UnifiedAlchemyMagicMock

Base = declarative_base()

class Item(Base):
    __tablename__ = 'items'
    id = Column(Integer, primary_key=True)
    name = Column(String)


@pytest.fixture
def real_db_session():
    engine = create_engine('sqlite:///real.db')

    with engine.connect() as conn:
        Session = sessionmaker(bind=conn)
        Base.metadata.create_all(engine)
        session = Session()

        sample_item = Item(name='fizz')
        session.add(sample_item)
        session.commit()

        yield session


@pytest.fixture
def mocked_db_session():
    session = UnifiedAlchemyMagicMock()
    session.add(Item(name='fizz'))
    return session


@pytest.fixture(params=('real', 'mock'))
def db_session(request, real_db_session, mocked_db_session):
    backend_type = request.param
    if backend_type == 'real':
        return real_db_session
    elif backend_type == 'mock':
        return mocked_db_session

测试示例：

def test_fizz(db_session):
    assert db_session.query(Item).one().name == 'fizz'

执行收益：

$ pytest -v 
======================================= test session starts ========================================
platform linux -- Python 3.6.8, pytest-4.4.2, py-1.8.0, pluggy-0.11.0
cachedir: .pytest_cache
rootdir: /home/hoefling/projects/private/stackoverflow/so-56558823
plugins: xdist-1.28.0, forked-1.0.2, cov-2.7.1
collected 2 items                                                                                  

test_spam.py::test_fizz[real] PASSED                                                         [ 50%]
test_spam.py::test_fizz[mock] PASSED                                                         [100%]

===================================== 2 passed in 0.18 seconds =====================================

示例：自定义执行顺序

您将需要实现一个自定义的pytest_collection_modifyitems挂钩，您可以在其中使用已收集的测试列表。例如，要先运行real测试，然后再运行其余测试：

# conftest.py

def pytest_collection_modifyitems(session, config, items):
    items.sort(key=lambda item: 'real' in item.name, reverse=True)

示例：收集并评估测试结果

此示例基于问题my answer的How can I access the overall test result of a pytest test run during runtime?。松散地跟随它：

# conftest.py

def pytest_sessionstart(session):
    session.results = dict()


@pytest.hookimpl(tryfirst=True, hookwrapper=True)
def pytest_runtest_makereport(item, call):
    outcome = yield
    result = outcome.get_result()

    if result.when == 'call':
        item.session.results[item] = result


@pytest.fixture(scope='session', autouse=True)
def compare_results(request):
    yield  # wait for all tests to finish

    results = request.session.results

    # partition test results into reals and mocks
    def partition(pred, coll):
        first, second = itertools.tee(coll)
        return itertools.filterfalse(pred, first), filter(pred, second)

    mocks, reals = partition(lambda item: item.name.endswith('[real]'), results.keys())
    # process test results in pairs
    by_name = operator.attrgetter('name')
    for real, mock in zip(sorted(reals, key=by_name), sorted(mocks, key=by_name)):
        if results[real].outcome != results[mock].outcome:
            pytest.fail(
                'A pair of tests has different outcomes:\n'
                f'outcome of {real.name} is {results[real].outcome}\n'
                f'outcome of {mock.name} is {results[mock].outcome}'
            )

当然，这只是一个存根。例如，如果在第一对具有不同结果的测试中比较将失败，并且结果dict键的分区也会为reals和mocks生成不均匀的列表（如果您具有未参数化的测试等）。

Pytest两次运行所有测试并比较模拟和真实的结果

1 个答案:

示例：自定义执行顺序

示例：收集并评估测试结果