Question

我已经在tensorflow-gpu旁边的计算机（Ubuntu 16.04）上安装了CUDA和CUDNN。

使用的版本：CUDA 10.0，CUDNN 7.6，Python 3.6，Tensorflow 1.14

这是nvidia-smi的输出，显示了视频卡的配置。

| NVIDIA-SMI 410.78       Driver Version: 410.78       CUDA Version: 10.0     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|===============================+======================+======================|
|   0  GeForce GTX 960M    On   | 00000000:02:00.0 Off |                  N/A |
| N/A   44C    P8    N/A /  N/A |    675MiB /  4046MiB |      0%   E. Process |
+-------------------------------+----------------------+----------------------+

+-----------------------------------------------------------------------------+
| Processes:                                                       GPU Memory |
|  GPU       PID   Type   Process name                             Usage      |
|=============================================================================|
|    0      1502      G   /usr/lib/xorg/Xorg                           363MiB |
|    0      3281      G   compiz                                        96MiB |
|    0      4375      G   ...uest-channel-token=14359313252217012722    69MiB |
|    0      5157      C   ...felipe/proj/venv/bin/python3.6            141MiB |
+-----------------------------------------------------------------------------+

这是device_lib.list_local_devices()的输出（tensorflow帮助器方法，以显示它可以看到的设备），表明我的GPU对tensorflow可见：

[name: "/device:CPU:0"
  device_type: "CPU"
  memory_limit: 268435456
  locality {
  }
  incarnation: 5096693727819965430, 
name: "/device:XLA_GPU:0"
  device_type: "XLA_GPU"
  memory_limit: 17179869184
  locality {
  }
  incarnation: 13415556283266501672
  physical_device_desc: "device: XLA_GPU device", 
name: "/device:XLA_CPU:0"
  device_type: "XLA_CPU"
  memory_limit: 17179869184
  locality {
  }
  incarnation: 14339781620792127180
  physical_device_desc: "device: XLA_CPU device", 
name: "/device:GPU:0"
  device_type: "GPU"
  memory_limit: 3464953856
  locality {
    bus_id: 1
    links {
    }
  }
  incarnation: 13743207545082600644
  physical_device_desc: "device: 0, name: GeForce GTX 960M, pci bus id: 0000:02:00.0, compute capability: 5.0"
]

现在将实际使用GPU进行计算。我使用一小段代码在CPU和GPU上运行一些虚拟矩阵乘法，以比较性能：

shapes = [(50, 50), (100, 100), (500, 500), (1000, 1000), (10000,10000), (15000,15000)]

devices = ['/device:CPU:0', '/device:XLA_GPU:0']

for device in devices:
    for shape in shapes:
        with tf.device(device):
            random_matrix = tf.random_uniform(shape=shape, minval=0, maxval=1)
            dot_operation = tf.matmul(random_matrix, tf.transpose(random_matrix))
            sum_operation = tf.reduce_sum(dot_operation)

        # Time the actual runtime of the operations
        start_time = datetime.now()
        with tf.Session(config=tf.ConfigProto(log_device_placement=True)) as session:
            result = session.run(sum_operation)
        elapsed_time = datetime.now() - start_time

        # PRINT ELAPSED TIME, SHAPE AND DEVICE USED

这是惊喜。第一次运行包含此代码块的单元时（我在jupyter笔记本上）， GPU计算所需的时间比CPU长得多：

# output of first run: CPU is faster
----------------------------------------
Input shape: (50, 50) using Device: /device:CPU:0 took: 0.01
Input shape: (100, 100) using Device: /device:CPU:0 took: 0.01
Input shape: (500, 500) using Device: /device:CPU:0 took: 0.01
Input shape: (1000, 1000) using Device: /device:CPU:0 took: 0.02
Input shape: (10000, 10000) using Device: /device:CPU:0 took: 6.22
Input shape: (15000, 15000) using Device: /device:CPU:0 took: 21.23
----------------------------------------
Input shape: (50, 50) using Device: /device:XLA_GPU:0 took: 2.82
Input shape: (100, 100) using Device: /device:XLA_GPU:0 took: 0.17
Input shape: (500, 500) using Device: /device:XLA_GPU:0 took: 0.18
Input shape: (1000, 1000) using Device: /device:XLA_GPU:0 took: 0.20
Input shape: (10000, 10000) using Device: /device:XLA_GPU:0 took: 28.36
Input shape: (15000, 15000) using Device: /device:XLA_GPU:0 took: 93.73
----------------------------------------

惊喜2 ：当我重新运行包含伪矩阵乘法代码的单元时，GPU版本要快得多（如预期）：

# output of reruns: GPU is faster
----------------------------------------
Input shape: (50, 50) using Device: /device:CPU:0 took: 0.02
Input shape: (100, 100) using Device: /device:CPU:0 took: 0.02
Input shape: (500, 500) using Device: /device:CPU:0 took: 0.02
Input shape: (1000, 1000) using Device: /device:CPU:0 took: 0.04
Input shape: (10000, 10000) using Device: /device:CPU:0 took: 6.78
Input shape: (15000, 15000) using Device: /device:CPU:0 took: 24.65
----------------------------------------
Input shape: (50, 50) using Device: /device:XLA_GPU:0 took: 0.14
Input shape: (100, 100) using Device: /device:XLA_GPU:0 took: 0.12
Input shape: (500, 500) using Device: /device:XLA_GPU:0 took: 0.13
Input shape: (1000, 1000) using Device: /device:XLA_GPU:0 took: 0.14
Input shape: (10000, 10000) using Device: /device:XLA_GPU:0 took: 1.64
Input shape: (15000, 15000) using Device: /device:XLA_GPU:0 took: 5.29
----------------------------------------

所以我的问题是：为什么只有运行代码一次才真正发生GPU加速？

我可以看到GPU正确设置（否则根本不会加速）。是由于某种初始开销造成的吗？在我们实际使用GPU之前，GPU是否需要进行预热？

？

PS：在这两次运行中（例如，GPU速度较慢，而下一个GPU速度较快），我可以看到GPU的使用率为100％，因此肯定是使用。

P.S。：只有在第一次运行时，GPU似乎才没有被拾取。如果我随后将其运行两次，三遍或多次，则所有运行都在第一个运行成功之后（即GPU计算更快）。

Answer 1

robert-crovella's comment使我研究了XLA，这有助于我找到解决方案。

可以通过两种方式将GPU映射到Tensorflow设备：作为XLA设备和作为普通GPU。

这就是为什么有两个设备的原因，一个叫"/device:XLA_GPU:0"，另一个叫"/device:GPU:0"。

我需要做的是改为激活"/device:GPU:0"。现在，GPU立即被Tensorflow拾取。

Tensorflow：GPU加速仅在首次运行后发生

1 个答案: