Question

Tensorflow默认分配所有GPU内存，但我的新设置实际上只有9588 MiB / 11264 MiB。我预计大约11.000MiB就像我的旧设置一样。

Tensorflow信息在这里：

$ from tensorflow.python.client import device_lib
$ print(device_lib.list_local_devices())

[name: "/cpu:0"
device_type: "CPU"
memory_limit: 268435456
locality {
}
incarnation: 9709578925658430097
, name: "/gpu:0"
device_type: "GPU"
memory_limit: 9273834701
locality {
  bus_id: 1
}
incarnation: 16668416364446126258
physical_device_desc: "device: 0, name: GeForce GTX 1080 Ti, pci bus id: 0000:03:00.0"
, name: "/gpu:1"
device_type: "GPU"
memory_limit: 9273834701
locality {
  bus_id: 1
}
incarnation: 2094938711079475130
physical_device_desc: "device: 1, name: GeForce GTX 1080 Ti, pci bus id: 0000:04:00.0"
]

nvidia-smi.exe说：

+-----------------------------------------------------------------------------+
| NVIDIA-SMI 385.41                 Driver Version: 385.41                    |
|-------------------------------+----------------------+----------------------+
| GPU  Name            TCC/WDDM | Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|===============================+======================+======================|
|   0  GeForce GTX 108... WDDM  | 00000000:03:00.0 Off |                  N/A |
| 23%   35C    P8    13W / 250W |   9284MiB / 11264MiB |      0%      Default |
+-------------------------------+----------------------+----------------------+
|   1  GeForce GTX 108... WDDM  | 00000000:04:00.0 Off |                  N/A |
| 23%   38C    P2    55W / 250W |   9146MiB / 11264MiB |      0%      Default |
+-------------------------------+----------------------+----------------------+

+-----------------------------------------------------------------------------+
| Processes:                                                       GPU Memory |
|  GPU       PID   Type   Process name                             Usage      |
|=============================================================================|
|    0      1280    C+G   ...mmersiveControlPanel\SystemSettings.exe N/A      |
|    0      1448      C   ...ers\Administrator\Anaconda3\pythonw.exe N/A      |
|    0      1560    C+G   Insufficient Permissions                   N/A      |
|    0      4120    C+G   ...6)\Google\Chrome\Application\chrome.exe N/A      |
|    0      4580    C+G   C:\Windows\explorer.exe                    N/A      |
|    0      5188    C+G   ...t_cw5n1h2txyewy\ShellExperienceHost.exe N/A      |
|    0      5324    C+G   ...dows.Cortana_cw5n1h2txyewy\SearchUI.exe N/A      |
|    1      1228    C+G   Insufficient Permissions                   N/A      |
|    1      1244    C+G   Insufficient Permissions                   N/A      |
|    1      1448      C   ...ers\Administrator\Anaconda3\pythonw.exe N/A      |
+-----------------------------------------------------------------------------+

我的环境是这样的：

操作系统：Windows10 library：python 3.6，keras 2.0.8，tensorflow-gpu 1.3.0，CUDA8.0 CUDNN6.0

有谁知道原因？

Answer 1

有必要使用TCC驱动程序来避免窗口保留一些VRAM。您可能正在使用WDDM驱动程序。

以下是TCC的页面： https://docs.nvidia.com/gameworks/content/developertools/desktop/nsight/tesla_compute_cluster.htm

以下是相关问题： How can I use 100% of VRAM on a secondary GPU from a single process on windows 10?

Answer 2

import tensorflow as tf
gpu_options = tf.GPUOptions(per_process_gpu_memory_fraction=0.2)
sess = tf.Session(config=tf.ConfigProto(gpu_options=gpu_options))
from keras import backend as K
import tensorflow as tf
config = tf.ConfigProto()
config.gpu_options.per_process_gpu_memory_fraction = 0.2
session = tf.Session(config=config)
K.set_session(session)

这适用于我的案例

Tensorflow不会分配完整的GPU内存

2 个答案: