Question

我按照tutorial来量化我的图形为8位。我不能在这里分享精确的图形，但我可以说它是一个简单的卷积神经网络。

当我在原始和量化网络上运行benchmark tool时，很明显量化网络要慢得多（100毫秒对4.5毫秒）。

原始网络中最慢的节点：

time average [ms]   [%] [cdf%]  [Op]    [Name]
1.198   26.54%  26.54%  MatMul  fc10/fc10/MatMul
0.337   7.47%   34.02%  Conv2D  conv2/Conv2D
0.332   7.36%   41.37%  Conv2D  conv4/Conv2D
0.323   7.15%   48.53%  Conv2D  conv3/Conv2D
0.322   7.14%   55.66%  Conv2D  conv5/Conv2D
0.310   6.86%   62.53%  Conv2D  conv1/Conv2D
0.118   2.61%   65.13%  Conv2D  conv2_1/Conv2D
0.105   2.32%   67.45%  MaxPool pool1

量化网络中最慢的节点：

time average [ms]   [%] [cdf%]  [Op]    [Name]
8.289   47.67%  47.67%  QuantizedMatMul fc10/fc10/MatMul_eightbit_quantized_bias_add
5.398   5.33%   53.00%  QuantizedConv2D conv5/Conv2D_eightbit_quantized_conv
5.248   5.18%   58.18%  QuantizedConv2D conv4/Conv2D_eightbit_quantized_conv
4.981   4.92%   63.10%  QuantizedConv2D conv2/Conv2D_eightbit_quantized_conv
4.908   4.85%   67.95%  QuantizedConv2D conv3/Conv2D_eightbit_quantized_conv
3.167   3.13%   71.07%  QuantizedConv2D conv5_1/Conv2D_eightbit_quantized_conv
3.049   3.01%   74.08%  QuantizedConv2D conv4_1/Conv2D_eightbit_quantized_conv
2.973   2.94%   77.02%  QuantizedMatMul fc11/MatMul_eightbit_quantized_bias_add

这是什么原因？我正在使用从源代码编译的tensorflow版本，没有支持gpu。

Answer 1

https://github.com/tensorflow/tensorflow/issues/2807

点击此处查看评论。似乎量化尚未针对x86进行优化。我的量化初始resnet v2运行速度也比原来慢。

为什么量化图推断比使用原始图需要更多的时间？

1 个答案: