Question

我创建了一个程序，对于各种分割int并返回double结果的方法，为0到20之间的每个可能的分子和分母组合运行此方法，包括0和20它给出了运行所有这些组合所需时间的结果：

Casting numerator:*-                                       0.000029 ms average (2852 ms / 100000000 reps)
Multiplying numerator:*-                                   0.000594 ms average (5940 ms / 10000000 reps)
Making double variable out of numerator:                   0.002192 ms average (2192 ms / 1000000 reps)
Casting denominator:*+                                     0.002201 ms average (2201 ms / 1000000 reps)
Multiplying denominator:                                   0.002181 ms average (2181 ms / 1000000 reps)
Making double variable out of denominator:                 0.002183 ms average (2183 ms / 1000000 reps)
Passing double numerator:*--                               0.000041 ms average (408 ms / 10000000 reps)
Passing double denominator:*--                             0.000034 ms average (343 ms / 10000000 reps)

Casting numerator, dividing twice:                         0.002506 ms average (2506 ms / 1000000 reps)
Multiplying numerator, dividing twice:                     0.002505 ms average (2505 ms / 1000000 reps)
Making double variable out of numerator, dividing twice:   0.002517 ms average (2517 ms / 1000000 reps)
Casting denominator, dividing twice:                       0.002520 ms average (2520 ms / 1000000 reps)
Multiplying denominator, dividing twice:                   0.002517 ms average (2517 ms / 1000000 reps)
Making double variable out of denominator, dividing twice: 0.002508 ms average (2508 ms / 1000000 reps)
Passing double numerator, dividing twice:*-                0.000704 ms average (7040 ms / 10000000 reps)
Passing double denominator, dividing twice:*-              0.000594 ms average (5935 ms / 10000000 reps)

但是，以不同的顺序运行这些测试会导致某些测试花费相同的时间（如预期的那样），但其他测试需要花费相同的时间（*表示我得到的结果明显不同）：< / p>

Casting numerator:                                         0.000029 ms average (2904 ms / 100000000 reps)
Casting denominator:                                       0.000726 ms average (7263 ms / 10000000 reps)
Multiplying numerator:                                     0.002220 ms average (2220 ms / 1000000 reps)
Multiplying denominator:                                   0.002224 ms average (2224 ms / 1000000 reps)
Making double variable out of numerator:                   0.002236 ms average (2236 ms / 1000000 reps)
Making double variable out of denominator:                 0.002249 ms average (2249 ms / 1000000 reps)
Passing double denominator:                                0.000036 ms average (3586 ms / 100000000 reps)
Passing double numerator:                                  0.001455 ms average (1455 ms / 1000000 reps)

Casting numerator, dividing twice:                         0.002542 ms average (2542 ms / 1000000 reps)
Casting denominator, dividing twice:                       0.002546 ms average (2546 ms / 1000000 reps)
Multiplying numerator, dividing twice:                     0.002545 ms average (2545 ms / 1000000 reps)
Multiplying denominator, dividing twice:                   0.002542 ms average (2542 ms / 1000000 reps)
Making double variable out of numerator, dividing twice:   0.002559 ms average (2559 ms / 1000000 reps)
Making double variable out of denominator, dividing twice: 0.002588 ms average (2588 ms / 1000000 reps)
Passing double denominator, dividing twice:                0.000700 ms average (7002 ms / 10000000 reps)
Passing double numerator, dividing twice:                  0.001581 ms average (1581 ms / 1000000 reps)

并且还运行一个“控制”测试，循环遍历所有可能的组合，但实际上并没有划分任何东西产生甚至更奇怪的结果，控制执行和数字投射曾经做过但分子现在做了20倍差，之后完成的控制比以前花费的时间多得多：

Control (Don't actually divide anything):                  0.000035 ms average (3494 ms / 100000000 reps)
Casting numerator:                                         0.000588 ms average (5880 ms / 10000000 reps)
Casting denominator:                                       0.002177 ms average (2177 ms / 1000000 reps)
Multiplying numerator:                                     0.002188 ms average (2188 ms / 1000000 reps)
Multiplying denominator:                                   0.002202 ms average (2202 ms / 1000000 reps)
Making double variable out of numerator:                   0.002186 ms average (2186 ms / 1000000 reps)
Making double variable out of denominator:                 0.002201 ms average (2201 ms / 1000000 reps)
Passing double denominator:                                0.002024 ms average (2024 ms / 1000000 reps)
Passing double numerator:                                  0.001456 ms average (1456 ms / 1000000 reps)

Control (Don't actually divide anything):                  0.000927 ms average (927 ms / 1000000 reps)
Casting numerator, dividing twice:                         0.002552 ms average (2552 ms / 1000000 reps)
Casting denominator, dividing twice:                       0.002556 ms average (2556 ms / 1000000 reps)
Multiplying numerator, dividing twice:                     0.002538 ms average (2538 ms / 1000000 reps)
Multiplying denominator, dividing twice:                   0.002554 ms average (2554 ms / 1000000 reps)
Making double variable out of numerator, dividing twice:   0.002546 ms average (2546 ms / 1000000 reps)
Making double variable out of denominator, dividing twice: 0.002535 ms average (2535 ms / 1000000 reps)
Passing double denominator, dividing twice:                0.002344 ms average (2344 ms / 1000000 reps)
Passing double numerator, dividing twice:                  0.001597 ms average (1597 ms / 1000000 reps)

这些结果对于重新运行相同的订单是一致的。改变重复次数通常对平均速度没有显着影响，但是在运行控制10 ^ 8次然后运行控制10 ^ 7次的情况下，我发现运行铸造分子旁边执行同样好的控制如果它如果运行10 ^ 6或10 ^ 8次，则运行10 ^ 7次，但运行速度要慢20倍。使程序等待输入并将其进程优先级设置为Windows中的Realtime没有显着差异。什么是JVM和/或CPU以不同的方式使这些性能如此不同？

更新：“运行预热测试”

重试第一个测试顺序和第三个测试顺序，其中每个测试连续运行六次，每次连续运行相同的测试需要大约相同的运行时间（最多+ - 15％，与之相比无关紧要）除了控制和转换分子之外，在第3到第6次连续测试中运行0 ms，并乘以分子，在第一个测试中需要0ms来运行分子。第4到第6次测试。此外，每个测试似乎花费的时间大约等于在第三个测试顺序中运行该测试一次所花费的时间。不知何故，在许多情况下，运行热身测试会使性能恶化！

测试程序

public class Tests {
    public static void main(String[] args) throws Exception {
        //number of repetitions set so that each takes 1-10 seconds total to run on my machine
        testPerformance(castNumerator, 8);
        testPerformance(multiplyNumerator, 7);
        testPerformance(makeNumerator, 6);
        testPerformance(castDenominator, 6);
        testPerformance(multiplyDenominator, 6);
        testPerformance(makeDenominator, 6);
        testPerformance(diDivider, 6);
        testPerformance(idDivider, 8);
        System.out.println();
        testPerformance(castNumerator2, 6);
        testPerformance(multiplyNumerator2, 6);
        testPerformance(makeNumerator2, 6);
        testPerformance(castDenominator2, 6);
        testPerformance(multiplyDenominator2, 6);
        testPerformance(makeDenominator2, 6);
        testPerformance(diDivider2, 6);
        testPerformance(idDivider2, 7);
    }

    static void testPerformance(final Divider divider, final int logReps) {
        final int reps = (int)Math.pow(10, logReps);
        final long startTime;
        if (divider instanceof IntIntDivider) {
            final IntIntDivider iiDivider = ((IntIntDivider)divider);
            startTime = System.currentTimeMillis();
            for (int i = 0; i < reps; i++) {
                for (int n = 0; n < 20; n++) {
                    for (int d = 0; d < 20; d++) {
                        iiDivider.divide(n, d);
                    }
                }
            }
        } else if (divider instanceof DoubleIntDivider) {//yucky repetition, but the only fair way to do it because generics can't do primitives
            final DoubleIntDivider diDivider = ((DoubleIntDivider)divider);
            startTime = System.currentTimeMillis();
            for (int i = 0; i < reps; i++) {
                for (int n = 0; n < 20; n++) {
                    for (int d = 0; d < 20; d++) {
                        diDivider.divide(n, d);
                    }
                }
            }
        } else if (divider instanceof IntDoubleDivider) {
            final IntDoubleDivider idDivider = ((IntDoubleDivider)divider);
            startTime = System.currentTimeMillis();
            for (int i = 0; i < reps; i++) {
                for (int n = 0; n < 20; n++) {
                    for (int d = 0; d < 20; d++) {
                        idDivider.divide(n, d);
                    }
                }
            }
        } else {
            throw new RuntimeException("Impossible divider");
        }
        final long endTime = System.currentTimeMillis();
        final long time = (endTime - startTime);
        System.out.printf("    %-58s %f ms%n", divider + ":", time / (double) reps); //cast reps to double because casting time might result in precision loss
        //System.out.println(divider + ":");
        //System.out.println("\t" + time + " ms taken for " + reps + " runs");
        //System.out.printf("\tAverage of %f ms%n", time / (double) reps);
    }
    static interface Divider {}
    static abstract class IntIntDivider implements Divider {
        public abstract double divide(int n, int d);
    }
    static abstract class DoubleIntDivider implements Divider {
        public abstract double divide(double n, int d);
    }
    static abstract class IntDoubleDivider implements Divider {
        public abstract double divide(int n, double d);
    }
    static final IntIntDivider control = new IntIntDivider() {
        @Override
        public double divide(int n, int d) {
            return 1;
        }
        @Override
        public String toString() {
            return "Control (Don't actually divide anything)";
        }
    };
    static final IntIntDivider castNumerator = new IntIntDivider() {
        @Override
        public double divide(int n, int d) {
            return ((double)n) / d;
        }
        @Override
        public String toString() {
            return "Casting numerator";
        }
    };
    static final IntIntDivider multiplyNumerator = new IntIntDivider() {
        @Override
        public double divide(int n, int d) {
            return n * 1.0 / d;
        }
        @Override
        public String toString() {
            return "Multiplying numerator";
        }
    };
    static final IntIntDivider makeNumerator = new IntIntDivider() {
        @Override
        public double divide(int n, int d) {
            final double nDouble = n;
            return nDouble / d;
        }
        @Override
        public String toString() {
            return "Making double variable out of numerator";
        }
    };
    static final IntIntDivider castDenominator = new IntIntDivider() {
        @Override
        public double divide(int n, int d) {
            return n / (double) d;
        }
        @Override
        public String toString() {
            return "Casting denominator";
        }
    };
    static final IntIntDivider multiplyDenominator = new IntIntDivider() {
        @Override
        public double divide(int n, int d) {
            return n / (d * 1.0);
        }
        @Override
        public String toString() {
            return "Multiplying denominator";
        }
    };
    static final IntIntDivider makeDenominator = new IntIntDivider() {
        @Override
        public double divide(int n, int d) {
            final double dDouble = d;
            return n / dDouble;
        }
        @Override
        public String toString() {
            return "Making double variable out of denominator";
        }
    };
    static final DoubleIntDivider diDivider = new DoubleIntDivider() {
        @Override
        public double divide(double n, int d) {
            return n / d;
        }
        @Override
        public String toString() {
            return "Passing double numerator";
        }
    };
    static final IntDoubleDivider idDivider = new IntDoubleDivider() {
        @Override
        public double divide(int n, double d) {
            return n / d;
        }
        @Override
        public String toString() {
            return "Passing double denominator";
        }
    };
    static final IntIntDivider castNumerator2 = new IntIntDivider() {
        @Override
        public double divide(int n, int d) {
            return ((double)n) / d + ((double)n) / d;
        }
        @Override
        public String toString() {
            return "Casting numerator, dividing twice";
        }
    };
    static final IntIntDivider multiplyNumerator2 = new IntIntDivider() {
        @Override
        public double divide(int n, int d) {
            return n * 1.0 / d + n * 1.0 / d;
        }
        @Override
        public String toString() {
            return "Multiplying numerator, dividing twice";
        }
    };
    static final IntIntDivider makeNumerator2 = new IntIntDivider() {

        @Override
        public double divide(int n, int d) {
            final double nDouble = n;
            return nDouble / d + nDouble / d;
        }
        @Override
        public String toString() {
            return "Making double variable out of numerator, dividing twice";
        }
    };
    static final IntIntDivider castDenominator2 = new IntIntDivider() {
        @Override
        public double divide(int n, int d) {
            return n / (double) d + n / (double) d;
        }
        @Override
        public String toString() {
            return "Casting denominator, dividing twice";
        }
    };
    static final IntIntDivider multiplyDenominator2 = new IntIntDivider() {
        @Override
        public double divide(int n, int d) {
            return n / (d * 1.0) + n / (d * 1.0);
        }
        @Override
        public String toString() {
            return "Multiplying denominator, dividing twice";
        }
    };
    static final IntIntDivider makeDenominator2 = new IntIntDivider() {
        @Override
        public double divide(int n, int d) {
            final double dDouble = d;
            return n / dDouble + n / dDouble;
        }
        @Override
        public String toString() {
            return "Making double variable out of denominator, dividing twice";
        }
    };
    static final DoubleIntDivider diDivider2 = new DoubleIntDivider() {
        @Override
        public double divide(double n, int d) {
            return n / d + n / d;
        }
        @Override
        public String toString() {
            return "Passing double numerator, dividing twice";
        }
    };
    static final IntDoubleDivider idDivider2 = new IntDoubleDivider() {
        @Override
        public double divide(int n, double d) {
            return n / d + n / d;
        }
        @Override
        public String toString() {
            return "Passing double denominator, dividing twice";
        }
    };
}

Answer 1

我怀疑订购在这里有效。问题是在更改排序的结果中，测试运行的次数也会发生变化。这是一个问题，因为代码运行的次数越多，JIT就会越优化它。

如果仅查看数字，那么一个非常强大的模式是，运行次数更多的测试更快。这是非常经典的JIT行为。优化的编译会运行更多次并权衡平均值。因此，应该清楚的是，单个测试运行的次数必须相等。此外，测试应考虑到热身。

因此调整main就是这样：

for(int i = 0; i < 2; i++) {
    testPerformance(castNumerator, 6);
    testPerformance(multiplyNumerator, 6);
    testPerformance(makeNumerator, 6);
    testPerformance(castDenominator, 6);
    testPerformance(multiplyDenominator, 6);
    testPerformance(makeDenominator, 6);
    testPerformance(diDivider, 6);
    testPerformance(idDivider, 6);
    System.out.println();
    testPerformance(castNumerator2, 6);
    testPerformance(multiplyNumerator2, 6);
    testPerformance(makeNumerator2, 6);
    testPerformance(castDenominator2, 6);
    testPerformance(multiplyDenominator2, 6);
    testPerformance(makeDenominator2, 6);
    testPerformance(diDivider2, 6);
    testPerformance(idDivider2, 6);
    System.out.println();
}

对于热身，这只是运行测试两次，第一组结果可能没有多大意义。

但在这样做之后，我在第二次运行中看到类似下面的内容：

Casting numerator:                                         0.003012 ms
Multiplying numerator:                                     0.003011 ms
Making double variable out of numerator:                   0.003025 ms
Casting denominator:                                       0.003040 ms
Multiplying denominator:                                   0.003015 ms
Making double variable out of denominator:                 0.003006 ms
Passing double numerator:                                  0.000000 ms
Passing double denominator:                                0.000000 ms

这有点奇怪，因为这表明最后两次测试需要0毫秒才能运行。并不是因为这些划分实际上非常快，因为更改程序使其使用nanoTime无效。（nanoTime总是被推荐，但是因为时间足够长，它可能不会产生很大的差别。）

实际上可能发生的事情是JIT已经发现你忽略了返回值，所以它通过不以某种方式对它们进行优化来优化这些测试。至于为什么它只对某些测试做，你的猜测可能和我的一样好，但它表明一个严重的问题。如果JIT正在进行这种极端优化，我们不知道它在其他地方做了什么。

这可以通过使用返回值执行某些操作（基本上任何事情）来解决：

// somewhere
static long result;

// for each op
result += idDivider.divide(n, d);

// at the end of the test
System.out.println(result);

这可能会减慢测试速度，但会影响JIT的聪明才智。

在所有这些变化之后，我每次都基本上得到这个：

Casting numerator:                                         0.007088 ms
Multiplying numerator:                                     0.007135 ms
Making double variable out of numerator:                   0.007162 ms
Casting denominator:                                       0.007180 ms
Multiplying denominator:                                   0.007206 ms
Making double variable out of denominator:                 0.007173 ms
Passing double numerator:                                  0.003650 ms
Passing double denominator:                                0.003663 ms

Casting numerator, dividing twice:                         0.007554 ms
Multiplying numerator, dividing twice:                     0.007574 ms
Making double variable out of numerator, dividing twice:   0.007538 ms
Casting denominator, dividing twice:                       0.007550 ms
Multiplying denominator, dividing twice:                   0.007503 ms
Making double variable out of denominator, dividing twice: 0.007577 ms
Passing double numerator, dividing twice:                  0.003765 ms
Passing double denominator, dividing twice:                0.003798 ms

现在，如果我不得不猜测为什么这表明它作为参数转换速度更快，我会说HotSpot正在编译它，以便循环变量总是一个双倍。

在基准测试中，我建议进一步阅读：

Robust Java benchmarking, Part 1: Issues
Java theory and practice: Anatomy of a flawed microbenchmark
Zed Shaw's crazy and convincing rant on poor statistics（对于任何容易被冒犯的人来说，他都是粗鲁的。）

为什么以不同的顺序运行我的测试会导致性能大不相同？

1 个答案: