Question

I like to include google benchmark tests within out continuous integration environment.

How can I check if the code under test has not become slower than it was before? Do I have to manually process the json output of the benchmark?

Is there an elegant way of accounting for different machines?

Is there an elegant way to check for every build on the same machine the performance? Is there a better way than writing a bash script and compare the output of the actual performance with a reference performance?

There has been a similar question, but no good answer to it.

Is there a way to integrate it into jenkins.

Thanks for your help. Best regards, Georg

Answer 1

要将Google Benchmark集成到Jenkins中，我使用了这个简洁的插件：

https://plugins.jenkins.io/benchmark

我的测试输出了.json文件

benchmarks.exe --benchmark_out=benchmarktest_output.json

并为插件编写了一个自定义JSON模式以了解输出：

{
  "description": "Google Benchmark JSON schema",
  "failure": { "value": true },
  "type": "object",
  "properties": {
    "benchmarks": {
      "type": "array",
      "items": {
        "type": "object",
        "properties": {
          "name": { "type": "name" },
          "real_time": { "type": "result" },
          "cpu_time": { "type": "result" },
          "iterations": { "type": "parameter" }
        }
      }
    }
  }
}

虽然我无法在插件中显示所有输出。例如，我还没有弄清楚如何显示时间单位以及真实时间和cpu时间，只能同时显示其中一个。

Answer 2

使用this python工具，可以自动评估Google基准测试，并将其集成到持续集成工具中

感谢@MikeVanDyke的提示。

How to include google benchmark in continuous integration

2 个答案: