是否可以用except捕获Python代码中的CUDA_ERROR_LAUNCH_TIMEOUT?

时间:2019-06-30 14:32:18

标签: python tensorflow memory cudnn

我设法在低级计算机上编译了CUDA-Tensorflow_Gpu(CUDA计算能力为3.0)。我在视频上运行模型文件,但有时会出现内存分配错误:

2019-06-30 17:11:15.525537: W tensorflow/core/common_runtime/bfc_allocator.cc:211] Allocator (GPU_0_bfc) ran out of memory trying to allocate 2.14GiB. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available.
2019-06-30 17:11:16.607461: W tensorflow/core/common_runtime/bfc_allocator.cc:211] Allocator (GPU_0_bfc) ran out of memory trying to allocate 3.11GiB. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available.
2019-06-30 17:11:18.926863: W tensorflow/core/common_runtime/bfc_allocator.cc:211] Allocator (GPU_0_bfc) ran out of memory trying to allocate 2.99GiB. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available.
2019-06-30 17:11:20.068998: W tensorflow/core/common_runtime/bfc_allocator.cc:211] Allocator (GPU_0_bfc) ran out of memory trying to allocate 3.32GiB. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available.
2019-06-30 17:11:25.334895: W tensorflow/core/common_runtime/bfc_allocator.cc:211] Allocator (GPU_0_bfc) ran out of memory trying to allocate 1.67GiB. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available.
2019-06-30 17:11:33.030001: E tensorflow/stream_executor/cuda/cuda_driver.cc:981] failed to synchronize the stop event: CUDA_ERROR_LAUNCH_TIMEOUT: the launch timed out and was terminated
2019-06-30 17:11:33.030044: E tensorflow/stream_executor/cuda/cuda_timer.cc:55] Internal: error destroying CUDA event in context 0x208cd570: CUDA_ERROR_LAUNCH_TIMEOUT: the launch timed out and was terminated
2019-06-30 17:11:33.030059: E tensorflow/stream_executor/cuda/cuda_timer.cc:60] Internal: error destroying CUDA event in context 0x208cd570: CUDA_ERROR_LAUNCH_TIMEOUT: the launch timed out and was terminated
2019-06-30 17:11:33.030101: F tensorflow/stream_executor/cuda/cuda_dnn.cc:231] Check failed: status == CUDNN_STATUS_SUCCESS (7 vs. 0)Failed to set cuDNN stream.
Aborted (core dumped)

我尝试在try-except块中执行代码,但无法捕获错误。我猜这个问题是关于在Python代码中捕获C ++库中发生的错误。有可能吗?

0 个答案:

没有答案