Question

我有一个带有单元测试的项目，当我运行ctest（macos上的cmake版本3.18.2）时，所有测试都报告成功。但是，如果我自己运行其中一个测试，则它的退出状态为1。据我所知，这不应该发生，那么是什么原因引起的，又该如何解决呢？

该问题是一个粗心的小错误，但是出于其他遇到该问题的人的利益，我提供了一个答案，可以跳过其余问题。

不幸的是，我无法使用更小的最小工作示例来重现此行为。我会尝试提供尽可能多的相关信息，如果我遗漏了一些东西，请告诉我。这是单元测试的代码：

#include "alignment_reader.h"

#define CATCH_CONFIG_MAIN
#define CATCH_CONFIG_COLOUR_NONE
#include "catch.h"

#include "string_conversions.h"

#include "exceptions.h"

namespace paste_alignments {

namespace test {

namespace {

SCENARIO("Test correctness of AlignmentReader::FromFile.",
         "[AlignmentReader][FromFile][correctness]") {

  GIVEN("The name of a valid input file.") {
    std::string input_file{"test_data/valid_alignment_file.tsv"};

    WHEN("Constructed with default number of fields and chunk size.") {
      CHECK_NOTHROW(AlignmentReader::FromFile(input_file));
      /*AlignmentReader reader{AlignmentReader::FromFile(input_file)};

      THEN("Field number and chunk size are at default values.") {
        CHECK(reader.NumFields() == 12);
        CHECK(reader.ChunkSize() == 128 * 1000 * 1000);
      }*/
    }
  }
}

} // namespace

} // namespace test

} // namespace paste_alignments

当我取消注释部分时，退出代码更改为2，并且ctest仍然报告成功。

这是我运行ctest时发生的情况（带有和不带有注释部分）：

$ ctest
Test project /Users/Jasper/cpp_projects/PasteAlignments/debug
    Start 1: alignment_reader_test
1/1 Test #1: alignment_reader_test ............   Passed    0.16 sec

100% tests passed, 0 tests failed out of 1

Total Test time (real) =   0.17 sec

如果我单独运行测试并检查退出状态（其中的内容已注释掉），这就是我得到的：

$ ./test/alignment_reader_test 

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
alignment_reader_test is a Catch v2.12.2 host application.
Run with -? for options

-------------------------------------------------------------------------------
Scenario: Test correctness of AlignmentReader::FromFile.
      Given: The name of a valid input file.
       When: Constructed with default number of fields and chunk size.
-------------------------------------------------------------------------------
/Users/Jasper/cpp_projects/PasteAlignments/test/alignment_reader_test.cc:64
...............................................................................

/Users/Jasper/cpp_projects/PasteAlignments/test/alignment_reader_test.cc:65: FAILED:
  CHECK_NOTHROW( AlignmentReader::FromFile(input_file) )
due to unexpected exception with message:
  Unable to open file: 'test_data/valid_alignment_file.tsv'.

===============================================================================
test cases: 1 | 1 failed
assertions: 1 | 1 failed

$ echo $?
1

当我添加注释部分时唯一改变的是两次测试均失败，并且退出状态为2而不是1。

此处带有--verbose标志：

$ ctest --verbose
UpdateCTestConfiguration  from :/Users/Jasper/cpp_projects/PasteAlignments/debug/DartConfiguration.tcl
Parse Config file:/Users/Jasper/cpp_projects/PasteAlignments/debug/DartConfiguration.tcl
UpdateCTestConfiguration  from :/Users/Jasper/cpp_projects/PasteAlignments/debug/DartConfiguration.tcl
Parse Config file:/Users/Jasper/cpp_projects/PasteAlignments/debug/DartConfiguration.tcl
Test project /Users/Jasper/cpp_projects/PasteAlignments/debug
Constructing a list of tests
Done constructing a list of tests
Updating test list for fixtures
Added 0 tests to meet fixture requirements
Checking test dependency graph...
Checking test dependency graph end
test 1
    Start 1: alignment_reader_test

1: Test command: /Users/Jasper/cpp_projects/PasteAlignments/debug/test/alignment_reader_test
1: Test timeout computed to be: 1500
1: ===============================================================================
1: All tests passed (1 assertion in 1 test case)
1: 
1/1 Test #1: alignment_reader_test ............   Passed    0.36 sec

100% tests passed, 0 tests failed out of 1

Total Test time (real) =   0.36 sec

我没有更改ctest配置（如建议的here）。 CMakeLists.txt看起来像这样：

add_executable(alignment_reader_test
        "${PROJECT_SOURCE_DIR}/test/alignment_reader_test.cc"
        "${PROJECT_SOURCE_DIR}/src/alignment_reader.cc"
        "${PROJECT_SOURCE_DIR}/src/alignment_batch.cc"
        "${PROJECT_SOURCE_DIR}/src/scoring_system.cc"
        "${PROJECT_SOURCE_DIR}/src/alignment.cc"
        "${PROJECT_SOURCE_DIR}/src/helpers.cc")
target_include_directories(alignment_reader_test PUBLIC
        "${PROJECT_SOURCE_DIR}/test"
        "${PROJECT_SOURCE_DIR}/include"
        "${PROJECT_SOURCE_DIR}/lib/catch/include")
add_test(NAME alignment_reader_test COMMAND alignment_reader_test)

Answer 1

由于@Tsyvarev，我发现了问题。

构建目录结构如下：

<script src="https://cdnjs.cloudflare.com/ajax/libs/d3/5.7.0/d3.min.js"></script>
<div id="gridchart"></div>

当CMake运行build/ |--test/ |--test_executable |--test_data/ |--test_datafile时，它将在目录test_executable中运行它。

当我单独运行可执行文件时，我从test目录运行了该可执行文件。（build）。

在单元测试代码中，./test/test_executable被称为test_datafile。当测试是从test_data/test_datafile可执行文件运行时（与我一样），而不是从build目录运行。

因此，当ctest运行测试时，它实际上应该成功完成。

实际上，如果我先build/test进入cd，则测试的退出代码为0。

ctest报告测试成功，退出状态为1

1 个答案: