我正在使用183个图像和批注进行袋鼠检测(https://github.com/experiencor/keras-yolo3)的数据集从头开始训练keras-yolo3(https://github.com/experiencor/kangaroo),并支持GPU(Quadro T2000)和32 GB RAM。该训练模型包含约61 M个参数。
输出错误消息是:
ResourceExhaustedError: 2 root error(s) found.
(0) Resource exhausted: OOM when allocating tensor with shape[4,64,208,208] and type float on /job:localhost/replica:0/task:0/device:GPU:0 by allocator GPU_0_bfc
[[{{node training_4/Adam/gradients/zeros_273}}]]
Hint: If you want to see a list of allocated tensors when OOM happens, add report_tensor_allocations_upon_oom to RunOptions for current allocation info.
[[loss_4/Identity_2/_3879]]
Hint: If you want to see a list of allocated tensors when OOM happens, add report_tensor_allocations_upon_oom to RunOptions for current allocation info.
(1) Resource exhausted: OOM when allocating tensor with shape[4,64,208,208] and type float on /job:localhost/replica:0/task:0/device:GPU:0 by allocator GPU_0_bfc
[[{{node training_4/Adam/gradients/zeros_273}}]]
Hint: If you want to see a list of allocated tensors when OOM happens, add report_tensor_allocations_upon_oom to RunOptions for current allocation info.
我不知道形状[4,64,208,208]的张量如何导致内存不足(OOM)。如何解决?