Question

假设我有一个提交文件 fileFromStudent.py ，其中唯一的内容是：

$ perl6 -e 'multi sub postfix:<Δ>(Int $n) { 137 }; say  6Δ;'
137

$ perl6 -e 'multi sub postfix:<Δ>(Int $n) { 137 }; my $ΔΔ = 6; say $ΔΔ;'
6

$ perl6 -e 'multi sub postfix:<Δ>(Int $n) { 137 }; my $Δ = 6; say $ΔΔ;'===SORRY!=== Error while compiling -e
Variable '$ΔΔ' is not declared
at -e:1
------> fix:<Δ>(Int $n) { 137 }; my $Δ = 6; say ⏏$ΔΔ;

我想测试stdout以查看学生是否正确写出了print语句。根据我的阅读，我已经能够创建以下代码：

print("hello world")

上述工作，但我是否正确的方式？我添加的其中一项内容是from io import StringIO from unittest.mock import patch import unittest, importlib, sys class TestStringMethods(unittest.TestCase): def setUp(self): studentSubmission = 'fileFromStudent' ## Stores output from print() in fakeOutput with patch('sys.stdout', new=StringIO()) as self.fakeOutput: ## Loads submission on first test, reloads on subsequent tests if studentSubmission in sys.modules: importlib.reload(sys.modules[ studentSubmission ] ) else: importlib.import_module( studentSubmission ) ## Test Cases def test_print_passes(self): test_case = "Checking Output Statement - Will Pass" self.output = self.fakeOutput.getvalue().strip() self.assertEqual(self.output, 'hello world', msg=test_case) def test_print_fails(self): test_case = "Checking Output Statement - Will Fail" self.output = self.fakeOutput.getvalue().strip() self.assertEqual(self.output, 'hell world', msg=test_case) if __name__ == '__main__': suite = unittest.TestLoader().loadTestsFromTestCase(TestStringMethods) testResult = unittest.TextTestRunner(verbosity=2).run(suite)调用以重新加载学生的程序。这是因为在最初几周，我将让学生使用print（）作为他们的最终输出（直到我们进入函数）。

我知道它看起来很模糊，或者为什么我应该打扰因为它有效，但是上面的代码是一个正确的方法来构建它还是我完全遗漏了使这一切变得简单的东西？

Answer 1

我花了几个星期的时间在这方面取得了成功，头痛和谷歌。我没有采用Popen路线的原因之一是我想捕捉学生提交的错误代码是否会立即崩溃。信不信由你，Intro课程的前几周就是这样。由于我发现的所有内容都是2011年至2012年，所以我认为我发布了此消息，以便Google未来能够找到它。

扩展我上面所写的内容，让我们假设下一个任务是获得一个输入并说出＆＃34;嗨＆＃34;

name = input("What's your name? ")
print("Hi " + name)

现在，我想自动化测试，看看我是否可以输入"Adam"并返回"Hi Adam"。为此，我选择使用StringIO作为我的标准输入（sys.stdin = StringIO("Adam")）。这使我可以控制文本流的来源和去向。另外，我不想看到学生可能发生的所有错误（sys.stderr = StringIO()）。

正如我所提到的，我选择使用importlib代替Popen。我想确保如果学生提交虚假代码而不是破坏所有内容，那么就无法通过我正在运行的任何测试。我尝试了subprocess和py.test，虽然它们可能更好，更清洁，但我无法找到任何对我如何使其正常运动有意义的事情。

以下是我最新版测试的副本：

from io import StringIO
from unittest.mock import patch
import unittest, importlib, sys, os
from time import sleep

# setup the environment
backup = sys.stderr

class TestTypingExercise(unittest.TestCase):
    def __init__(self, test_name, filename, inputs):
        super(TestTypingExercise, self).__init__(test_name)
        self.library = filename.split('.')[0]
        self.inputs = inputs

    def setUp(self):
        sys.stdin = StringIO(self.inputs[0])
        try:
            ## Stores output from print() in fakeOutput
            with patch('sys.stdout', new=StringIO()) as self.fakeOutput:
                ## Loads submission on first test, reloads on subsequent tests
                if self.library in sys.modules:
                    importlib.reload(sys.modules[ self.library ] )
                else:
                    importlib.import_module( self.library )
        except Exception as e:
            self.fail("Failed to Load - {0}".format(str(e)))

    ## Test Cases
    def test_code_runs(self):
        test_case = "Checking to See if code can run"
        self.assertTrue(True, msg=test_case)

    def test_says_hello(self):
        test_case = "Checking to See if code said 'Hi Adam'"
        # Regex might be cleaner, but this typically solves most cases
        self.output = self.fakeOutput.getvalue().strip().lower()
        self.assertTrue('hi adam' in self.output, msg=test_case)

if __name__ == '__main__':
    ignore_list = ["grader.py"]

    # Run Through Each Submitted File
    directory = os.listdir('.')
    for filename in sorted(directory):
        if (filename.split('.')[-1] != 'py') or (filename in ignore_list):
            continue
        #print("*"*15, filename, "*"*15)

        # 'Disables' stderr, so I don't have to see all their errors
        sys.stderr = StringIO()     # capture output

        # Run Tests Across Student's Submission
        suite = unittest.TestSuite()
        suite.addTest(TestTypingExercise('test_code_runs', filename, 'Adam'))
        suite.addTest(TestTypingExercise('test_says_hello', filename, 'Adam'))
        results = unittest.TextTestRunner().run(suite)

        # Reset stderr
        out = sys.stderr.getvalue() # release output
        sys.stderr.close()  # close the stream 
        sys.stderr = backup # restore original stderr

        # Display Test Results
        print(filename,"Test Results - ", end='')
        if not results.wasSuccessful():
            print("Failed (test cases that failed):")
            for error in results.failures:
                print('\t',error[1].split('\n')[-2])
        else:
            print("Pass!")
        sleep(0.05)

以下是最终结果：

StudentSubmission01.py Test Results - Failed (test cases that failed):
     AssertionError: Failed to Load - EOL while scanning string literal (StudentSubmission01.py, line 23)
     AssertionError: Failed to Load - EOL while scanning string literal (StudentSubmission01.py, line 23)
StudentSubmission02.py Test Results - Pass!
StudentSubmission03.py Test Results - Pass!
StudentSubmission04.py Test Results - Pass!
StudentSubmission05.py Test Results - Pass!
StudentSubmission06.py Test Results - Pass!
StudentSubmission07.py Test Results - Pass!
StudentSubmission08.py Test Results - Pass!
StudentSubmission09.py Test Results - Pass!
StudentSubmission10.py Test Results - Pass!
StudentSubmission11.py Test Results - Pass!
StudentSubmission12.py Test Results - Pass!
StudentSubmission13.py Test Results - Pass!
[Finished in 0.9s]

如果我想测试多个不同的输入，我可能需要移动一些东西，但现在这样可行。

这是用Python 3 unittest测试stdout的正确方法吗？

1 个答案: