在Python的单元测试中将元数据添加到TestCase

时间:2018-09-28 17:11:41

标签: python nunit python-unittest

我想将元数据添加到我编写的使用Python的unittest框架的TestCase中的各个测试中。元数据(实际上是字符串)需要在测试过程中携带并输出到XML文件。

除了保留测试之外,单元测试和我的测试代码都不会使用数据。 (我有一个程序将在以后运行,打开XML文件,然后查找元数据/字符串。)

我以前使用过NUnit,它允许使用C#属性来执行此操作。具体来说,您可以将其放在类之上:

[Property("SmartArrayAOD", -3)]

,然后在XML输出中找到它。

是否可以在Python的单元测试中将元数据附加到测试?

1 个答案:

答案 0 :(得分:0)

仅转储XML的简单方法

如果您要做的只是在每次单元测试后将内容写入XML文件,只需在测试类中添加tearDown方法(例如,如果您拥有, give it a)。

class MyTest(unittest.TestCase):
    def tearDown(self):
        dump_xml_however_you_do()

    def test_whatever(self):
        pass

常规方法

如果您希望采用一种通用的方式来收集和跟踪所有测试中的元数据并在最后将其返回,请尝试在测试类的__init__()中创建一个星形表,并在tearDown()期间向其中添加行,然后从unittest中提取对测试类的初始化实例的引用,如下所示:

第1步:设置unittest.TestCase的可重用子类,这样我们就不必重复表处理了

(将所有示例代码放入同一文件或复制导入)

"""
Demonstration of adding and retrieving meta data from python unittest tests
"""

import sys
import warnings
import unittest
import copy
import time
import astropy
import astropy.table
if sys.version_info < (3, 0):
    from StringIO import StringIO
else:
    from io import StringIO


class DemoTest(unittest.TestCase):
    """
    Demonstrates setup of an astropy table in __init__, adding data to the table in tearDown
    """

    def __init__(self, *args, **kwargs):
        super(DemoTest, self).__init__(*args, **kwargs)

        # Storing results in a list made it convenient to aggregate them later
        self.results_tables = [astropy.table.Table(
            names=('Name', 'Result', 'Time', 'Notes'),
            dtype=('S50', 'S30', 'f8', 'S50'),
        )]
        self.results_tables[0]['Time'].unit = 'ms'
        self.results_tables[0]['Time'].format = '0.3e'

        self.test_timing_t0 = 0
        self.test_timing_t1 = 0

    def setUp(self):
        self.test_timing_t0 = time.time()

    def tearDown(self):
        test_name = '.'.join(self.id().split('.')[-2:])
        self.test_timing_t1 = time.time()
        dt = self.test_timing_t1 - self.test_timing_t0

        # Check for errors/failures in order to get state & description.  https://stackoverflow.com/a/39606065/6605826
        if hasattr(self, '_outcome'):  # Python 3.4+
            result = self.defaultTestResult()  # these 2 methods have no side effects
            self._feedErrorsToResult(result, self._outcome.errors)
            problem = result.errors or result.failures
            state = not problem
            if result.errors:
                exc_note = result.errors[0][1].split('\n')[-2]
            elif result.failures:
                exc_note = result.failures[0][1].split('\n')[-2]
            else:
                exc_note = ''
        else:  # Python 3.2 - 3.3 or 3.0 - 3.1 and 2.7
            # result = getattr(self, '_outcomeForDoCleanups', self._resultForDoCleanups)  # DOESN'T WORK RELIABLY
            # This is probably only good for python 2.x, meaning python 3.0, 3.1, 3.2, 3.3 are not supported.
            exc_type, exc_value, exc_traceback = sys.exc_info()
            state = exc_type is None
            exc_note = '' if exc_value is None else '{}: {}'.format(exc_type.__name__, exc_value)

        # Add a row to the results table
        self.results_tables[0].add_row()
        self.results_tables[0][-1]['Time'] = dt*1000  # Convert to ms
        self.results_tables[0][-1]['Result'] = 'pass' if state else 'FAIL'
        with warnings.catch_warnings():
            warnings.filterwarnings('ignore', category=astropy.table.StringTruncateWarning)
            self.results_tables[0][-1]['Name'] = test_name
            self.results_tables[0][-1]['Notes'] = exc_note

步骤2:设置测试管理器以提取元数据

def manage_tests(tests):
    """
    Function for running tests and extracting meta data
    :param tests: list of classes sub-classed from DemoTest

    :return: (TextTestResult, Table, string)
        result returned by unittest
        astropy table
        string: formatted version of the table

    """
    table_sorting_columns = ['Result', 'Time']

    # Build test suite
    suite_list = []
    for test in tests:
        suite_list.append(unittest.TestLoader().loadTestsFromTestCase(test))
    combo_suite = unittest.TestSuite(suite_list)

    # Run tests
    results = [unittest.TextTestRunner(verbosity=1, stream=StringIO(), failfast=False).run(combo_suite)]

    # Catch test classes
    suite_tests = []
    for suite in suite_list:
        suite_tests += suite._tests

    # Collect results tables
    results_tables = []
    for suite_test in suite_tests:
        if getattr(suite_test, 'results_tables', [None])[0] is not None:
            results_tables += copy.copy(suite_test.results_tables)

    # Process tables, if any
    if len(results_tables):
        a = []
        while (len(a) == 0) and len(results_tables):
            a = results_tables.pop(0)  # Skip empty tables, if any
        results_table = a
        for rt in results_tables:
            if len(rt):
                with warnings.catch_warnings():
                    warnings.filterwarnings('ignore', category=DeprecationWarning)
                    results_table = astropy.table.join(results_table, rt, join_type='outer')
        try:
            results_table = results_table.group_by(table_sorting_columns)
        except Exception:
            print('Error sorting test results table. Columns may not be in the preferred order.')
        column_names = list(results_table.columns.keys())
        alignments = ['<' if cn == 'Notes' else '>' for cn in column_names]
        if len(results_table):
            rtf = '\n'.join(results_table.pformat(align=alignments, max_width=-1))
            exp_res = sum([result.testsRun - len(result.skipped) for result in results])
            if len(results_table) != exp_res:
                print('ERROR forming results table. Expected {} results, but table length is {}.'.format(
                    exp_res, len(results_table),
                ))
        else:
            rtf = None

    else:
        results_table = rtf = None

    return results, results_table, rtf

第3步:用法示例

class FunTest1(DemoTest):
    @staticmethod
    def test_pass_1():
        pass

    @staticmethod
    def test_fail_1():
        assert False, 'Meant to fail for demo 1'


class FunTest2(DemoTest):
    @staticmethod
    def test_pass_2():
        pass

    @staticmethod
    def test_fail_2():
        assert False, 'Meant to fail for demo 2'


res, tab, form = manage_tests([FunTest1, FunTest2])
print(form)
print('')
for r in res:
    print(r)
    for error in r.errors:
        print(error[0])
        print(error[1])

样本结果:

$ python unittest_metadata.py 
        Name         Result    Time                    Notes                  
                                ms                                            
-------------------- ------ --------- ----------------------------------------
FunTest2.test_fail_2   FAIL 5.412e-02 AssertionError: Meant to fail for demo 2
FunTest1.test_fail_1   FAIL 1.118e-01 AssertionError: Meant to fail for demo 1
FunTest2.test_pass_2   pass 6.199e-03                                         
FunTest1.test_pass_1   pass 6.914e-03                                         

<unittest.runner.TextTestResult run=4 errors=0 failures=2>

应该与python 2.7或3.7一起使用。您可以将任何所需的列添加到表中。您可以在setUptearDown中甚至在测试过程中向表中添加参数和填充。

警告:

此解决方案访问_tests的受保护属性unittest.suite.TestSuite,可能会产生意外结果。这个特定的实现可以像我在python2.7和python3.7中所预期的那样工作,但是在套件的构建和询问方式上的细微变化很容易导致奇怪的事情发生。不过,我找不到其他方法来提取对unittest使用的类实例的引用。