Is there any way of "swapping" GPU memory in tensorflow?

Question

I am training a large model which exceeds the GPU memory (more than 11 GB). I am wondering if there are any way in tensorflow to swap the GPU memory into main memory. Some loss of efficiency is acceptable. Training the model fully on CPU solves the problem of memory but is too slow.

Answer 1

tensorflow中有一些类用 swap_memory 参数创建网络。

e.g。对于RNN，您可以使用 tf.nn.dynamic_rnn 或 tf.nn.raw_rnn

此参数还有一个更通用的循环类 tf.while_loop 。

但是使用内存交换没有一般选择。

只需查看tensorflow.org并使用其搜索功能即可。您可以使用 swap_memory

找到相关的类