Question

Julia 1.0.0 documentation提供了一般性提示。

它还建议您不要使用@time宏：

要进行更严格的基准测试，请考虑BenchmarkTools.jl软件包，该软件包除其他功能外还多次评估该功能，以降低噪声。

它们在使用中如何进行比较？使用不在“基本” Julia中的东西值得麻烦吗？

Answer 1

从统计角度来看，@ benchmark比@time好得多

TL; DR BenchmarkTools @benchmark宏是一个很棒的微基准测试工具。小心使用@time宏，不要认真对待第一轮。

这个简单的例子说明了用法和区别：

julia> # Fresh Julia 1.0.0 REPL

julia> # Add BenchmarkTools package using ] key package manager

(v1.0) pkg> add BenchmarkTools  
julia> # Press backspace key to get back to Julia REPL

# Load BenchmarkTools package into current REPL
julia> using BenchmarkTools

julia> # Definine a function with a known elapsed time
julia> f(n) = sleep(n)  # n is in seconds
f (generic function with 1 method)

# Expect just over 500 ms for elapsed time
julia> @benchmark f(0.5)
BenchmarkTools.Trial:
  memory estimate:  192 bytes
  allocs estimate:  5
  --------------
  minimum time:     501.825 ms (0.00% GC)
  median time:      507.386 ms (0.00% GC)
  mean time:        508.069 ms (0.00% GC)
  maximum time:     514.496 ms (0.00% GC)
  --------------
  samples:          10
  evals/sample:     1

julia> # Try second run to compare consistency
julia> # Note the very close consistency in ms for both median and mean times

julia> @benchmark f(0.5)
BenchmarkTools.Trial:
  memory estimate:  192 bytes
  allocs estimate:  5
  --------------
  minimum time:     502.603 ms (0.00% GC)
  median time:      508.716 ms (0.00% GC)
  mean time:        508.619 ms (0.00% GC)
  maximum time:     515.602 ms (0.00% GC)
  --------------
  samples:          10
  evals/sample:     1


julia> # Define the same function with new name for @time macro tests
julia> g(n) = sleep(n)
g (generic function with 1 method)

# First run suffers from compilation time, so 518 ms
julia> @time sleep(0.5)
  0.517897 seconds (83 allocations: 5.813 KiB)

# Second run drops to 502 ms, 16 ms drop
julia> @time sleep(0.5)
  0.502038 seconds (9 allocations: 352 bytes)

# Third run similar to second
julia> @time sleep(0.5)
  0.503606 seconds (9 allocations: 352 bytes)

# Fourth run increases over second by about 13 ms
julia> @time sleep(0.5)
  0.514629 seconds (9 allocations: 352 bytes)

这个简单的示例说明了使用@benchmark宏有多么容易，并谨慎使用了@time宏结果。

是的，使用@benchmark宏是值得的麻烦。

如何在Julia中进行正确的微基准测试？

1 个答案: