High cpu utilization while using 3D convolution on gpu in theano 0.9

Question

My cpu utilization is 100% when using theano.tensor.nnet.conv3d I am adding conv3d_fft, convgrad3d_fft and convtransp3d_fft to my compiling mode in the theano function. The interesting part is my gpu utilization is also high. My data is all on gpu memory.

Any ideas about why it cpu utilization is so high?

Thanks!

Update: I tried the same convolution in Keras with Theano backend. It is shockingly faster than theano despite the fact that it also uses conv3d.

Answer 1

为了在GPU上调用任务，CPU必须做一些工作（不可忽略）。如果GPU上的任务很小，则CPU上的任务变得相对较大。 顺便说一句，内存传输由DMA完成，除了调用传输外不会导致CPU负载。

我有一个CUDA程序，其中CPU每〜100μs调用一次GPU任务。分析我发现的程序，它是CPU绑定的，除了调用GPU任务外没有在CPU上完成任何操作。