使用pytest,为什么单个测试结果与运行所有测试不同?

时间:2018-06-26 19:21:44

标签: python unit-testing pytest

使用pytest和Python 3.6,我正在为项目进行管道仿真。我有测试可用于不同的阀门对准等。

奇怪的是,如果我在目录中运行所有测试,那么所有测试都会通过。如果我尝试运行一个测试类,它将失败。我尝试添加设置/拆卸方法以将其与其余代码分开,以防万一它受到其他类的影响。

这是测试代码( ffc 是组件库, fff 是方法库):

class TestGate3TankLevels:
    @classmethod
    def setup_class(cls):
        ffc.tank2.level = 18.0
        fff.change_tank_level(ffc.tank2, ffc.tank2.level)
        fff.gate1_open()
        fff.gate2_open()
        fff.gate3_open()
        fff.gate4_open()

    def test_gate3_tank_levels(self):
        assert ffc.gate1.position == 100
        assert ffc.gate1.flow_out == 19542.86939891452
        assert ffc.gate1.press_out == 13.109851301499999

        assert ffc.gate2.position == 100
        assert ffc.gate2.flow_out == 19542.86939891452
        assert ffc.gate2.press_out == 6.5549256507499996

        assert ffc.gate3.position == 100
        assert ffc.gate3.flow_in == 19542.86939891452
        assert ffc.gate3.press_in == 13.109851301499999
        assert ffc.gate3.flow_out == 19542.86939891452
        assert ffc.gate3.press_out == 13.109851301499999
        assert ffc.gate6.flow_in == 19542.86939891452
        assert ffc.gate6.press_in == 13.109851301499999

        assert ffc.gate4.position == 100
        assert ffc.gate4.flow_in == 0.0
        assert ffc.gate4.press_in == 0.0
        assert ffc.gate4.flow_out == 0.0
        assert ffc.gate4.press_out == 0.0

    @classmethod
    def teardown_class(cls):
        ffc.tank2.level = 32.0
        fff.change_tank_level(ffc.tank2, ffc.tank2.level)

单独运行时,我得到以下结果:

F
test_fuel_components.py:408 (TestGate3TankLevels.test_gate3_tank_levels)
19542.86939891452 != 39085.73879782904

Expected :39085.73879782904
Actual   :19542.86939891452
 <Click to see difference>

self = <VirtualPLC.tests.models.fuel_farm.test_fuel_components.TestGate3TankLevels object at 0x7f9b35289a20>

    def test_gate3_tank_levels(self):
        assert ffc.gate1.position == 100
        assert ffc.gate1.flow_out == 19542.86939891452
        assert ffc.gate1.press_out == 13.109851301499999

        assert ffc.gate2.position == 100
        assert ffc.gate2.flow_out == 19542.86939891452
        assert ffc.gate2.press_out == 6.5549256507499996

        assert ffc.gate3.position == 100
        assert ffc.gate3.flow_in == 19542.86939891452
        assert ffc.gate3.press_in == 13.109851301499999
        assert ffc.gate3.flow_out == 19542.86939891452
        assert ffc.gate3.press_out == 13.109851301499999
>       assert ffc.gate6.flow_in == 19542.86939891452
E       assert 39085.73879782904 == 19542.86939891452
E        +  where 39085.73879782904 = <PipingSystems.valve.valve.Gate object at 0x7f9b3515ce80>.flow_in
E        +    where <PipingSystems.valve.valve.Gate object at 0x7f9b3515ce80> = ffc.gate6

test_fuel_components.py:423: AssertionError

但是,如果我在test_fuel_components.py文件中运行所有测试,那么我一切都通过了:

Testing started at 2:16 PM ...
/home/cody/PycharmProjects/VirtualPLC/venv/bin/python /home/cody/.local/share/JetBrains/Toolbox/apps/PyCharm-P/ch-0/182.3341.8/helpers/pycharm/_jb_pytest_runner.py --path /home/cody/PycharmProjects/VirtualPLC/tests/models/fuel_farm/test_fuel_components.py
Launching py.test with arguments /home/cody/PycharmProjects/VirtualPLC/tests/models/fuel_farm/test_fuel_components.py in /home/cody/PycharmProjects/VirtualPLC/tests/models/fuel_farm

============================= test session starts ==============================
platform linux -- Python 3.6.3, pytest-3.6.2, py-1.5.3, pluggy-0.6.0
rootdir: /home/cody/PycharmProjects/VirtualPLC/tests/models/fuel_farm, inifile:collected 20 items

test_fuel_components.py ....................                             [100%]

========================== 20 passed in 0.16 seconds ===========================
Process finished with exit code 0

如果我将此测试类移动到文件中的其他位置(现在它在所有其他测试的末尾),则它将导致所有后续测试失败以及自身失败。

我不知道测试类如何会自行失败,但是在与其他测试一起运行时还可以。

编辑 Link to other tests(太久了,无法在此处发布)

2 个答案:

答案 0 :(得分:2)

所以这里的问题是,当储罐模型为imported时,每个储罐和阀门的状态都会初始化一次,就像这里的tank1一样:

# Storage tanks
# Assumes 36 ft tall tank w/ 1 million gallon capacity = 27778 gallons per foot
# Assumes 16 inch diam transfer piping
tank1 = tank.Tank("Tank 1", level=36.0, fluid_density=DENSITY, spec_gravity=SPEC_GRAVITY, outlet_diam=16,
                  outlet_slope=0.25)
tank1.static_tank_press = tank1.level
tank1.gravity_flow(tank1.pipe_diam, tank1.pipe_slope, tank1.pipe_coeff)

在不了解该项目应该做什么的情况下,很难对这种设计提出具体的批评,但是这种设计具有大量的全局状态,这使得测试极为困难。

不管现有设计的优缺点,如果您希望测试可靠且独立,您需要做的就是在每次测试开始时将每个储罐的状态重置为已知状态。鉴于这里涉及多少全球状态,您的选择非常有限。

当您在全局存储了如此多的状态时,我的建议是在每次测试后使用setUptearDown方法重置为已知状态。

这未经测试,但是您正在寻找类似这样的东西:

original_states = {}
tanks = [tank1, tank2, tank3, tank4, tank5, tank6]
tank_attrs = [ "name", "level", "fluid_density", "spec_grav",
               "tank_press", "flow_out", "pipe_diam", "pipe_slope" ]
def setUp(self):
    for tank in self.tanks:
        self.original_states[id(tank)] = {}
        for attr in self.tank_attrs:
            self.original_states[id(tank)] = getattr(tank, attr)

def tearDown(self):
    try:
        for tank in self.tanks:
            for attr, value in self.original_states[id(tank)].items():
                setattr(tank, attr, value)
    finally:
        self.original_states = {}

基本上在每次测试之前存储原始状态,然后在每次测试之后将其恢复。这些属性中有些不可思议,因此可能要经过一些反复试验才能使其100%起作用。

答案 1 :(得分:0)

问题是储罐的整体状态。一开始我进行了一些测试,这些测试改变了其中一个坦克的水位,但从未将其恢复到正常状态。

通过在开始其余测试之前添加一些函数调用并确保储罐液位“满”,它解决了所有问题:

    fff.change_tank_level(ffc.tank1, 36)
    fff.change_tank_level(ffc.tank2, 36)

其中36是满载坦克。

因此,当我进入TestGate3TankLevels类时,更改储罐水平可以正常工作。然后,拆解方法会将水箱水位恢复到正常水平。