编辑：切换到一个更好的例子，并阐明了为什么这是一个真正的问题。
我想在Python中编写单元测试，在断言失败时继续执行，这样我就可以在单个测试中看到多个失败。例如：
class Car(object):
def __init__(self, make, model):
self.make = make
self.model = make # Copy and paste error: should be model.
self.has_seats = True
self.wheel_count = 3 # Typo: should be 4.
class CarTest(unittest.TestCase):
def test_init(self):
make = "Ford"
model = "Model T"
car = Car(make=make, model=model)
self.assertEqual(car.make, make)
self.assertEqual(car.model, model) # Failure!
self.assertTrue(car.has_seats)
self.assertEqual(car.wheel_count, 4) # Failure!
此处，测试的目的是确保Car __init__正确设置其字段。我可以将其分解为四种方法（这通常是一个好主意），但在这种情况下，我认为将它作为测试单个概念的单个方法（“对象初始化正确”）更具可读性。
如果我们假设最好不要破解方法，那么我遇到了一个新问题：我无法立即看到所有错误。当我修复model错误并重新运行测试时，会出现wheel_count错误。我第一次运行测试时，可以节省时间看两个错误。
为了比较，Google的C ++单元测试框架distinguishes between介于非致命EXPECT_*断言和致命ASSERT_*断言之间：
断言成对出现，测试相同但对当前函数有不同的影响。 ASSERT_ *版本在失败时会生成致命的故障，并中止当前的功能。 EXPECT_ *版本生成非致命故障，不会中止当前功能。通常EXPECT_ *是首选，因为它们允许在测试中报告多个失败。但是，如果在有问题的断言失败时没有意义继续，那么你应该使用ASSERT_ *。
有没有办法获得EXPECT_* - 就像Python unittest中的行为一样？如果不在unittest中，那么是否有另一个支持此行为的Python单元测试框架？
顺便说一下，我很好奇有多少现实测试可能会从非致命断言中受益，所以我看了一些code examples（编辑2014-08-19使用搜索代码而不是谷歌代码搜索，RIP ）。在第一页中随机选择的10个结果中，所有结果都包含在同一测试方法中进行多个独立断言的测试。所有人都会受益于非致命的断言。
答案 0 :(得分：37)
使用非致命断言的另一种方法是捕获断言异常并将异常存储在列表中。然后声明该列表是空的，作为tearDown的一部分。
import unittest
class Car(object):
def __init__(self, make, model):
self.make = make
self.model = make # Copy and paste error: should be model.
self.has_seats = True
self.wheel_count = 3 # Typo: should be 4.
class CarTest(unittest.TestCase):
def setUp(self):
self.verificationErrors = []
def tearDown(self):
self.assertEqual([], self.verificationErrors)
def test_init(self):
make = "Ford"
model = "Model T"
car = Car(make=make, model=model)
try: self.assertEqual(car.make, make)
except AssertionError, e: self.verificationErrors.append(str(e))
try: self.assertEqual(car.model, model) # Failure!
except AssertionError, e: self.verificationErrors.append(str(e))
try: self.assertTrue(car.has_seats)
except AssertionError, e: self.verificationErrors.append(str(e))
try: self.assertEqual(car.wheel_count, 4) # Failure!
except AssertionError, e: self.verificationErrors.append(str(e))
if __name__ == "__main__":
unittest.main()
答案 1 :(得分：26)
一个选项是将所有值一次作为元组断言。
例如：
class CarTest(unittest.TestCase):
def test_init(self):
make = "Ford"
model = "Model T"
car = Car(make=make, model=model)
self.assertEqual(
(car.make, car.model, car.has_seats, car.wheel_count),
(make, model, True, 4))
此测试的输出为：
======================================================================
FAIL: test_init (test.CarTest)
----------------------------------------------------------------------
Traceback (most recent call last):
File "C:\temp\py_mult_assert\test.py", line 17, in test_init
(make, model, True, 4))
AssertionError: Tuples differ: ('Ford', 'Ford', True, 3) != ('Ford', 'Model T', True, 4)
First differing element 1:
Ford
Model T
- ('Ford', 'Ford', True, 3)
? ^ - ^
+ ('Ford', 'Model T', True, 4)
? ^ ++++ ^
这表明模型和车轮计数都不正确。
答案 2 :(得分：9)
在单个单元测试中具有多个断言被认为是反模式。单个单元测试预计只测试一件事。也许你的测试太多了。考虑将此测试分成多个测试。这样您就可以正确命名每个测试。
然而，有时可以同时检查多个事物。例如，当您断言同一对象的属性时。在这种情况下，您实际上断言该对象是否正确。一种方法是编写一个自定义帮助器方法，该方法知道如何在该对象上断言。您可以编写该方法，使其显示所有失败的属性，或者例如显示期望对象的完整状态以及断言失败时实际对象的完整状态。
答案 3 :(得分：8)
您可能想要做的是派生unittest.TestCase，因为那是断言失败时抛出的类。您将不得不重新设计您的TestCase以不抛出（可能会保留失败列表）。重新构建东西可能会导致您必须解决的其他问题。例如，您最终可能需要派生TestSuite进行更改，以支持对TestCase所做的更改。
答案 4 :(得分：8)
从Python 3.4开始，您还可以使用subtests：
def test_init(self):
make = "Ford"
model = "Model T"
car = Car(make=make, model=model)
with self.subTest(msg='Car.make check'):
self.assertEqual(car.make, make)
with self.subTest(msg='Car.model check'):
self.assertEqual(car.model, model)
with self.subTest(msg='Car.has_seats check'):
self.assertTrue(car.has_seats)
with self.subTest(msg='Car.wheel_count check'):
self.assertEqual(car.wheel_count, 4)
（{msg参数用于更轻松地确定哪个测试失败。）
输出：
======================================================================
FAIL: test_init (__main__.CarTest) [Car.model check]
----------------------------------------------------------------------
Traceback (most recent call last):
File "test.py", line 23, in test_init
self.assertEqual(car.model, model)
AssertionError: 'Ford' != 'Model T'
- Ford
+ Model T
======================================================================
FAIL: test_init (__main__.CarTest) [Car.wheel_count check]
----------------------------------------------------------------------
Traceback (most recent call last):
File "test.py", line 27, in test_init
self.assertEqual(car.wheel_count, 4)
AssertionError: 3 != 4
----------------------------------------------------------------------
Ran 1 test in 0.001s
FAILED (failures=2)
答案 5 :(得分：6)
使用单独的方法执行每个断言。
class MathTest(unittest.TestCase):
def test_addition1(self):
self.assertEqual(1 + 0, 1)
def test_addition2(self):
self.assertEqual(1 + 1, 3)
def test_addition3(self):
self.assertEqual(1 + (-1), 0)
def test_addition4(self):
self.assertEqaul(-1 + (-1), -1)
答案 6 :(得分：2)
我喜欢@ Anthony-Batchelor的方法来捕获AssertionError异常。但是这种使用装饰器的方法略有不同，也是一种通过/传递报告测试用例的方法。
#!/usr/bin/env python
# -*- coding: utf-8 -*-
import unittest
class UTReporter(object):
'''
The UT Report class keeps track of tests cases
that have been executed.
'''
def __init__(self):
self.testcases = []
print "init called"
def add_testcase(self, testcase):
self.testcases.append(testcase)
def display_report(self):
for tc in self.testcases:
msg = "=============================" + "\n" + \
"Name: " + tc['name'] + "\n" + \
"Description: " + str(tc['description']) + "\n" + \
"Status: " + tc['status'] + "\n"
print msg
reporter = UTReporter()
def assert_capture(*args, **kwargs):
'''
The Decorator defines the override behavior.
unit test functions decorated with this decorator, will ignore
the Unittest AssertionError. Instead they will log the test case
to the UTReporter.
'''
def assert_decorator(func):
def inner(*args, **kwargs):
tc = {}
tc['name'] = func.__name__
tc['description'] = func.__doc__
try:
func(*args, **kwargs)
tc['status'] = 'pass'
except AssertionError:
tc['status'] = 'fail'
reporter.add_testcase(tc)
return inner
return assert_decorator
class DecorateUt(unittest.TestCase):
@assert_capture()
def test_basic(self):
x = 5
self.assertEqual(x, 4)
@assert_capture()
def test_basic_2(self):
x = 4
self.assertEqual(x, 4)
def main():
#unittest.main()
suite = unittest.TestLoader().loadTestsFromTestCase(DecorateUt)
unittest.TextTestRunner(verbosity=2).run(suite)
reporter.display_report()
if __name__ == '__main__':
main()
来自控制台的输出：
(awsenv)$ ./decorators.py
init called
test_basic (__main__.DecorateUt) ... ok
test_basic_2 (__main__.DecorateUt) ... ok
----------------------------------------------------------------------
Ran 2 tests in 0.000s
OK
=============================
Name: test_basic
Description: None
Status: fail
=============================
Name: test_basic_2
Description: None
Status: pass
答案 7 :(得分：1)
期望在gtest中非常有用。 这是gist中的python方式，代码：
import sys
import unittest
class TestCase(unittest.TestCase):
def run(self, result=None):
if result is None:
self.result = self.defaultTestResult()
else:
self.result = result
return unittest.TestCase.run(self, result)
def expect(self, val, msg=None):
'''
Like TestCase.assert_, but doesn't halt the test.
'''
try:
self.assert_(val, msg)
except:
self.result.addFailure(self, sys.exc_info())
def expectEqual(self, first, second, msg=None):
try:
self.failUnlessEqual(first, second, msg)
except:
self.result.addFailure(self, sys.exc_info())
expect_equal = expectEqual
assert_equal = unittest.TestCase.assertEqual
assert_raises = unittest.TestCase.assertRaises
test_main = unittest.main
答案 8 :(得分：1)
PyPI中有一个名为softest的软断言包，可以满足您的要求。它通过收集故障，组合异常和堆栈跟踪数据并将其全部报告为常规unittest输出的一部分来工作。
例如，此代码：
import softest
class ExampleTest(softest.TestCase):
def test_example(self):
# be sure to pass the assert method object, not a call to it
self.soft_assert(self.assertEqual, 'Worf', 'wharf', 'Klingon is not ship receptacle')
# self.soft_assert(self.assertEqual('Worf', 'wharf', 'Klingon is not ship receptacle')) # will not work as desired
self.soft_assert(self.assertTrue, True)
self.soft_assert(self.assertTrue, False)
self.assert_all()
if __name__ == '__main__':
softest.main()
...产生此控制台输出：
======================================================================
FAIL: "test_example" (ExampleTest)
----------------------------------------------------------------------
Traceback (most recent call last):
File "C:\...\softest_test.py", line 14, in test_example
self.assert_all()
File "C:\...\softest\case.py", line 138, in assert_all
self.fail(''.join(failure_output))
AssertionError: ++++ soft assert failure details follow below ++++
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
The following 2 failures were found in "test_example" (ExampleTest):
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Failure 1 ("test_example" method)
+--------------------------------------------------------------------+
Traceback (most recent call last):
File "C:\...\softest_test.py", line 10, in test_example
self.soft_assert(self.assertEqual, 'Worf', 'wharf', 'Klingon is not ship receptacle')
File "C:\...\softest\case.py", line 84, in soft_assert
assert_method(*arguments, **keywords)
File "C:\...\Python\Python36-32\lib\unittest\case.py", line 829, in assertEqual
assertion_func(first, second, msg=msg)
File "C:\...\Python\Python36-32\lib\unittest\case.py", line 1203, in assertMultiLineEqual
self.fail(self._formatMessage(msg, standardMsg))
File "C:\...\Python\Python36-32\lib\unittest\case.py", line 670, in fail
raise self.failureException(msg)
AssertionError: 'Worf' != 'wharf'
- Worf
+ wharf
: Klingon is not ship receptacle
+--------------------------------------------------------------------+
Failure 2 ("test_example" method)
+--------------------------------------------------------------------+
Traceback (most recent call last):
File "C:\...\softest_test.py", line 12, in test_example
self.soft_assert(self.assertTrue, False)
File "C:\...\softest\case.py", line 84, in soft_assert
assert_method(*arguments, **keywords)
File "C:\...\Python\Python36-32\lib\unittest\case.py", line 682, in assertTrue
raise self.failureException(msg)
AssertionError: False is not true
----------------------------------------------------------------------
Ran 1 test in 0.000s
FAILED (failures=1)
注意：我创建并维护了softest。
答案 9 :(得分：0)
我认为没有办法用PyUnit做到这一点，并且不希望看到PyUnit以这种方式扩展。
我更喜欢每个测试函数（or more specifically asserting one concept per test）坚持一个断言，并将test_addition()重写为四个单独的测试函数。这将提供有关失败的更多有用信息， viz ：
.FF.
======================================================================
FAIL: test_addition_with_two_negatives (__main__.MathTest)
----------------------------------------------------------------------
Traceback (most recent call last):
File "test_addition.py", line 10, in test_addition_with_two_negatives
self.assertEqual(-1 + (-1), -1)
AssertionError: -2 != -1
======================================================================
FAIL: test_addition_with_two_positives (__main__.MathTest)
----------------------------------------------------------------------
Traceback (most recent call last):
File "test_addition.py", line 6, in test_addition_with_two_positives
self.assertEqual(1 + 1, 3) # Failure!
AssertionError: 2 != 3
----------------------------------------------------------------------
Ran 4 tests in 0.000s
FAILED (failures=2)
如果您认为此方法不适合您，您可能会发现this answer有帮助。
看起来您正在使用更新后的问题测试两个概念，我会将这些概念拆分为两个单元测试。第一个是在创建新对象时存储参数。这将有两个断言，一个用于make，另一个用于model。如果第一次失败，那显然需要修复，无论第二次通过还是失败都是无关紧要的。
第二个概念更值得怀疑......您正在测试是否初始化了一些默认值。的为什么？在实际使用它们时测试这些值会更有用（如果它们没有被使用，那么为什么它们在那里？）。
这两个测试都失败了，两者都应该。当我进行单元测试时，我对失败的兴趣远远超过我的成功，因为这是我需要集中注意力的地方。
FF
======================================================================
FAIL: test_creation_defaults (__main__.CarTest)
----------------------------------------------------------------------
Traceback (most recent call last):
File "test_car.py", line 25, in test_creation_defaults
self.assertEqual(self.car.wheel_count, 4) # Failure!
AssertionError: 3 != 4
======================================================================
FAIL: test_creation_parameters (__main__.CarTest)
----------------------------------------------------------------------
Traceback (most recent call last):
File "test_car.py", line 20, in test_creation_parameters
self.assertEqual(self.car.model, self.model) # Failure!
AssertionError: 'Ford' != 'Model T'
----------------------------------------------------------------------
Ran 2 tests in 0.000s
FAILED (failures=2)
答案 10 :(得分：0)
我对@Anthony Batchelor的答案有疑问，因为它迫使我在单元测试中使用try...catch。然后，我将try...catch逻辑封装在TestCase.assertEqual方法的替代中。以下技巧从单元测试代码中删除了try...catch块：
import unittest
import traceback
class AssertionErrorData(object):
def __init__(self, stacktrace, message):
super(AssertionErrorData, self).__init__()
self.stacktrace = stacktrace
self.message = message
class MultipleAssertionFailures(unittest.TestCase):
def __init__(self, *args, **kwargs):
self.verificationErrors = []
super(MultipleAssertionFailures, self).__init__( *args, **kwargs )
def tearDown(self):
super(MultipleAssertionFailures, self).tearDown()
if self.verificationErrors:
index = 0
errors = []
for error in self.verificationErrors:
index += 1
errors.append( "%s\nAssertionError %s: %s" % (
error.stacktrace, index, error.message ) )
self.fail( '\n\n' + "\n".join( errors ) )
self.verificationErrors.clear()
def assertEqual(self, goal, results, msg=None):
try:
super( MultipleAssertionFailures, self ).assertEqual( goal, results, msg )
except unittest.TestCase.failureException as error:
goodtraces = self._goodStackTraces()
self.verificationErrors.append(
AssertionErrorData( "\n".join( goodtraces[:-2] ), error ) )
def _goodStackTraces(self):
"""
Get only the relevant part of stacktrace.
"""
stop = False
found = False
goodtraces = []
# stacktrace = traceback.format_exc()
# stacktrace = traceback.format_stack()
stacktrace = traceback.extract_stack()
# https://stackoverflow.com/questions/54499367/how-to-correctly-override-testcase
for stack in stacktrace:
filename = stack.filename
if found and not stop and \
not filename.find( 'lib' ) < filename.find( 'unittest' ):
stop = True
if not found and filename.find( 'lib' ) < filename.find( 'unittest' ):
found = True
if stop and found:
stackline = ' File "%s", line %s, in %s\n %s' % (
stack.filename, stack.lineno, stack.name, stack.line )
goodtraces.append( stackline )
return goodtraces
# class DummyTestCase(unittest.TestCase):
class DummyTestCase(MultipleAssertionFailures):
def setUp(self):
self.maxDiff = None
super(DummyTestCase, self).setUp()
def tearDown(self):
super(DummyTestCase, self).tearDown()
def test_function_name(self):
self.assertEqual( "var", "bar" )
self.assertEqual( "1937", "511" )
if __name__ == '__main__':
unittest.main()
结果输出：
F
======================================================================
FAIL: test_function_name (__main__.DummyTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
File "D:\User\Downloads\test.py", line 77, in tearDown
super(DummyTestCase, self).tearDown()
File "D:\User\Downloads\test.py", line 29, in tearDown
self.fail( '\n\n' + "\n\n".join( errors ) )
AssertionError:
File "D:\User\Downloads\test.py", line 80, in test_function_name
self.assertEqual( "var", "bar" )
AssertionError 1: 'var' != 'bar'
- var
? ^
+ bar
? ^
:
File "D:\User\Downloads\test.py", line 81, in test_function_name
self.assertEqual( "1937", "511" )
AssertionError 2: '1937' != '511'
- 1937
+ 511
:
更多有关正确堆栈跟踪捕获的替代解决方案可以发布在How to correctly override TestCase.assertEqual(), producing the right stacktrace?
上答案 11 :(得分：0)
我知道这个问题是在几年前提出的，但是现在（至少）有两个Python软件包可让您执行此操作。
其中一个最柔软：https://pypi.org/project/softest/
另一个是Python延迟声明：https://github.com/pr4bh4sh/python-delayed-assert
我也没有用过，但是它们看起来和我非常相似。