示例代码在CUDNN上运行良好,但使用通过pip返回安装的代码无法创建cudnn句柄:CUDNN_STATUS_INTERNAL_ERROR。
我正在使用通过GPU 2080(x2)编译的Tensorflow 1.12,CUDA 10.0和CUDNN 7.4.1
这是Ubuntu 16.04 LTS。我也使用venv。一切都使用Python 3.5.2。在venv中完成。
我正在尝试使用https://pypi.org/project/py-agender/中的Py-Agender
使用GPU标志编译CUDA并成功验证它是否有效后,我尝试:
enter code here
pip3安装py-agender。
运行它会导致内部CUDNN错误。
我已按照其他回复的建议删除了〜/ .nv,但没有帮助
我还确认了CuDNN实际上可以使用GPU:
print(device_lib.list_local_devices()) returns:
2019-01-03 09:31:30.353768: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:964] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2019-01-03 09:31:30.354169: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1432] Found device 0 with properties:
name: GeForce RTX 2080 major: 7 minor: 5 memoryClockRate(GHz): 1.8
pciBusID: 0000:01:00.0
...所以我们至少有一些可行的方法!
b。 mNIST样本运行并返回预期值。
代码部分如下:
agender = PyAgender()
def get_gender(value) :
if value > 0.5:
return 'female'
else :
return 'male'
input_dir = sys.argv[1]
directories = os.listdir(input_dir)
if len(directories):
for video in directories:
files = glob.glob(input_dir + video + '/*')
if len(files):
for file_path in files:
filename, file_extension = os.path.splitext(os.path.basename(file_path))
faces = agender.detect_genders_ages(cv2.imread(file_path))
该代码仅在使用CPU编译的Tensorflow上运行良好。使用GPU,完整的错误是:
$ python age-gender.py ./frames/part1/
Using TensorFlow backend.
- Began to analyze the age and gender
2019-01-03 09:44:24.229618: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:964] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2019-01-03 09:44:24.230019: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1432] Found device 0 with properties:
name: GeForce RTX 2080 major: 7 minor: 5 memoryClockRate(GHz): 1.8
pciBusID: 0000:01:00.0 totalMemory: 7.76GiB freeMemory: 7.62GiB
2019-01-03 09:44:24.302145: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:964] su
ccessful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2019-01-03 09:44:24.302602: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1432] Found
device 1 with properties:name: GeForce RTX 2080 major: 7 minor: 5 memoryClockRate(GHz): 1.8 pciBusID: 0000:02:00.0 totalMemory: 7.77GiB freeMemory: 7.62GiB
2019-01-03 09:44:24.302625: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1511] Addin
g visible gpu devices: 0, 1
2019-01-03 09:44:24.665465: I tensorflow/core/common_runtime/gpu/gpu_device.cc:982] Device
interconnect StreamExecutor with strength 1 edge matrix:
2019-01-03 09:44:24.665492: I tensorflow/core/common_runtime/gpu/gpu_device.cc:988] 0 1
2019-01-03 09:44:24.665499: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1001] 0: N Y
2019-01-03 09:44:24.665504: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1001] 1: Y N
2019-01-03 09:44:24.665749: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1115] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 7335 MB memory) ->
physical GPU (device: 0, name: GeForce RTX 2080, pci bus id: 0000:01:00.0, compute capability: 7.5)
2019-01-03 09:44:24.666056: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1115] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:1 with 7338 MB memory) ->
physical GPU (device: 1, name: GeForce RTX 2080, pci bus id: 0000:02:00.0, compute capability: 7.5)
2019-01-03 09:44:25S :: Filename: ./frames/part1/2017/658.jpg
2019-01-03 09:44:25S :: Filename: ./frames/part1/2017/698.jpg
2019-01-03 09:44:25S :: Filename: ./frames/part1/2017/746.jpg
2019-01-03 09:44:25S :: Filename: ./frames/part1/2017/371.jpg
2019-01-03 09:44:25S :: Filename: ./frames/part1/2017/218.jpg
2019-01-03 09:44:25S :: Filename: ./frames/part1/2017/860.jpg
2019-01-03 09:44:26.702699: E tensorflow/stream_executor/cuda/cuda_dnn.cc:373] Could not create cudnn handle: CUDNN_STATUS_INTERNAL_ERROR
2019-01-03 09:44:26.709821: E tensorflow/stream_executor/cuda/cuda_dnn.cc:373] Could not create cudnn handle: CUDNN_STATUS_INTERNAL_ERROR
Traceback (most recent call last):
File "age-gender.py", line 50, in <module>
faces = agender.detect_genders_ages(cv2.imread(file_path))
File "/home/sv/.local/lib/python3.5/site-packages/pyagender/pyagender.py", line 56, in detect_genders_ages
height=face['height'])
File "/home/sv/.local/lib/python3.5/site-packages/pyagender/pyagender.py", line 82, in gender_age
result = self.resnet.predict(np.array([test_img]))
File "/opt/inta/venv/lib/python3.5/site-packages/keras/engine/training.py", line 1169, in predict
steps=steps)
File "/opt/inta/venv/lib/python3.5/site-packages/keras/engine/training_arrays.py", line 294, in predict_loop
batch_outs = f(ins_batch)
File "/opt/inta/venv/lib/python3.5/site-packages/keras/backend/tensorflow_backend.py", line 2715, in __call__
return self._call(inputs)
File "/opt/inta/venv/lib/python3.5/site-packages/keras/backend/tensorflow_backend.py", line 2675, in _call
fetched = self._callable_fn(*array_vals)
File "/opt/inta/venv/lib/python3.5/site-packages/tensorflow/python/client/session.py", line 1439, in __call__
run_metadata_ptr)
File "/opt/inta/venv/lib/python3.5/site-packages/tensorflow/python/framework/errors_impl.py", line 528, in __exit__
c_api.TF_GetCode(self.status.status))
tensorflow.python.framework.errors_impl.UnknownError: Failed to get convolution algorithm. This is probably because cuDNN failed to initialize, so try looking to see if a warning log message was printed above.
[[{{node conv2d_1/convolution}} = Conv2D[T=DT_FLOAT, _class=["loc:@batch_normalization_1/cond/FusedBatchNorm/Switch"], data_format="NCHW", dilations=[1, 1, 1, 1], padding="SAME", strides=[1, 1, 1, 1], use_cudnn_on_gpu=true, _device="/job:localhost/replica:0/task:0/device:GPU:0"](conv2d_1/convolution-0-TransposeNHWCToNCHW-LayoutOptimizer, conv2d_1/kernel/read)]]