如何使用QuickCheck调试差异测试

时间:2019-11-06 22:48:09

标签: haskell ghc quickcheck

我有一些使用Megaparsec的解析代码,我已经编写了一个简单的属性进行测试(它会生成一个随机表达式树,对其进行漂亮打印,然后检查结果是否可以解析回原始树)。

不幸的是,这似乎是一个错误,如果我不受限制地运行测试,我会看到GHC进程分配了越来越多的内存,直到我杀死它或OOM杀手先到达那里。

我认为这不是问题,但是我无法一生找出导致分歧的原因。该属性本身看起来像这样:(我已经剔除了适当的测试和缩小的代码,以尽量减少实际运行的代码)

prop_parse_expr :: Property
prop_parse_expr =
  forAll arbitrary $
  (\ pe ->
     let str = prettyParExpr 0 pe in
       counterexample ("Rendered to: " ++ show str) $
       trace ("STARTING TEST: " ++ show str) $
       case parse (expr <* eof) "" str of
         Left _ -> trace "NOPE" $ False
         Right _ -> trace "GOOD" $ True)

如果使用性能分析进行编译(使用stack test --profile),则可以使用RTS选项运行生成的二进制文件。啊,我想,并与-xc一起运行,以为将SIGINT发送给被卡住的作业时会得到有用的堆栈跟踪。好像没有与

一起运行
./long/path/to/foo-test -j1 --test-seed 1 +RTS -xc

我看到以下输出:

STARTING TEST: "0"
GOOD
STARTING TEST: "(x [( !0 )]) "
STARTING TEST: "({ 2 {( !0 )}} ) "
STARTING TEST: "{ 2{ ( x[0? {( 0) ,( x ) } :((0 )? (x ):0) -: ( -(^( x  )) ) ]), 0**( x )} } "
STARTING TEST: "| (0? (x[({ 1{ (0)? x : ( 0 ) }} ) ]) :(~&( 0) ?( x):( (x ) ^( x ) )))"
STARTING TEST: "(0 )"
STARTING TEST: "0"
^C*** Exception (reporting due to +RTS -xc): (THUNK_STATIC), stack trace: 
  Test.Framework.Improving.runImprovingIO,
  called from Test.Framework.Providers.QuickCheck2.runProperty,
  called from Test.Framework.Providers.QuickCheck2.runTest,
  called from Test.Framework.Runners.Core.runSimpleTest,
  called from Test.Framework.Runners.Core.runTestTree.go,
  called from Test.Framework.Runners.Core.runTestTree,
  called from Test.Framework.Runners.Core.runTests',
  called from Test.Framework.Runners.Core.runTests,
  called from Test.Framework.Runners.Console.defaultMainWithOpts,
  called from Test.Framework.Runners.Console.defaultMainWithArgs,
  called from Test.Framework.Runners.Console.defaultMain,
  called from Main.main
<snip: 2 more identical backtraces>
*** Exception (reporting due to +RTS -xc): (THUNK_STATIC), stack trace: 
  Test.Framework.Runners.Console.Utilities.hideCursorDuring,
  called from Test.Framework.Runners.Console.Run.showRunTestsTop,
  called from Test.Framework.Improving.runImprovingIO,
  called from Test.Framework.Providers.QuickCheck2.runProperty,
  called from Test.Framework.Providers.QuickCheck2.runTest,
  called from Test.Framework.Runners.Core.runSimpleTest,
  called from Test.Framework.Runners.Core.runTestTree.go,
  called from Test.Framework.Runners.Core.runTestTree,
  called from Test.Framework.Runners.Core.runTests',
  called from Test.Framework.Runners.Core.runTests,
  called from Test.Framework.Runners.Console.defaultMainWithOpts,
  called from Test.Framework.Runners.Console.defaultMainWithArgs,
  called from Test.Framework.Runners.Console.defaultMain,
  called from Main.main

谁能告诉我:

  1. 为什么我看到多条STARTING TEST行之间没有GOODNOPE,尽管有-j1?

  2. 我如何获得实际的堆栈跟踪信息,以显示测试在哪里分配所有内存?

感谢任何想法!

1 个答案:

答案 0 :(得分:0)

对于任何发现此问题的人,我的代码存在的问题是我的arbitrary表达式实例没有适当地限制大小,因此有时尝试制作大树。有关我应该做的事情,请参见QuickCheck manual的“生成递归数据类型”部分!

我发现正在运行以下命令:

./long/path/to/foo-test -o3 +RTS -xc

帮助我弄清楚发生了什么。奇怪的是,回溯仍然显示了几个执行线程。我真的不明白为什么,但是至少我可以看到我在“ makeAnExpr”函数中花费了时间。诀窍是调整超时时间(超过3秒),以使其在正常运行并真正卡住之前不会终止测试,但是会在测试开始消耗所有RAM之前停止测试!