根据Tensorflow的官方网站,Tensorflow功能默认使用GPU计算。
“如果TensorFlow操作同时具有CPU和GPU实现,则在将操作分配给设备时,GPU设备将获得优先权。”
我正在训练一个带有3层LSTM细胞的动态rnn。但在监控GPU使用情况时,我发现GPU负载为0%。
我的GPU是Nvidia GTX 960 M.详情
我google了很多但仍然没有发现任何东西。我很确定我安装了Tensorflow的gpu支持版本,并且已经更新了。 我想知道dynamic_rnn或LSTMCell是否没有GPU实现?有没有办法实现它?
感谢。
我的代码:
import numpy as np
import tensorflow as tf
import operator
lr = 0.001
seq_length= 50
char_per_iter = 5
n_inputs = 97
n_hidden_units = 700
n_layers = 3
keep_prob = tf.placeholder(tf.float32)
seq_length_holder = tf.placeholder(tf.int32)
str = open('C:/Users/david_000/workspace/RNN/text_generator/PandP.txt', 'r', encoding = 'utf8').read()
str = str[667:]
str = str.replace("\n", " ")
str = str.replace("‘", "'")
str = str.replace("’", "'")
str = str.replace("“", '"')
str = str.replace("”", '"')
# check if all char's are ascii
uni = "".join(set(str))
for ch in uni:
if ord(ch) >= 128:
print(ch)
x = []
for i in range(len(str)//char_per_iter):
x.append(str[i*char_per_iter : i*char_per_iter + seq_length])
def oneHot(char):
asc = ord(char)
asc -= 31
if asc < 97:
return np.eye(97)[asc].reshape(1,1,-1)
else:
return np.eye(97)[0].reshape(1,1,-1)
def getOneHot(seq):
out = []
for char in seq:
out.append(oneHot(char))
return np.array(out).reshape(1, len(seq), n_inputs)
'''
RNN
'''
# tf Graph input
xs = tf.placeholder(tf.float32, [1, None, n_inputs])
ys = tf.placeholder(tf.float32, [1, None, n_inputs])
def lstm_cell():
cell = tf.contrib.rnn.BasicLSTMCell(n_hidden_units)
cell = tf.contrib.rnn.DropoutWrapper(cell, output_keep_prob = keep_prob)
return cell
def RNN(X):
lstm_stacked = tf.contrib.rnn.MultiRNNCell([lstm_cell() for _ in range(n_layers)])
outputs, final_state = tf.nn.dynamic_rnn(lstm_stacked, X, dtype=tf.float32)
output = tf.layers.dense(outputs, n_inputs, activation=tf.nn.softmax)
output = tf.reshape(output, [-1, seq_length_holder, n_inputs])
return output
pred = RNN(xs)
cost = tf.losses.sigmoid_cross_entropy(ys, pred)
optimizer = tf.train.AdamOptimizer(lr)
train_step = optimizer.minimize(cost)
config = tf.ConfigProto()
config.gpu_options.allow_growth = True
sess = tf.Session(config=config)
init = tf.global_variables_initializer()
sess.run(init)
for i in range(47, len(x)):
sess.run(train_step, feed_dict = {xs : getOneHot(x[i]), ys : getOneHot(x[i+1]), keep_prob : 0.7, seq_length_holder : seq_length})
if i % 10 == 0:
print(i)
更新
现在我知道问题是Tensorlow无法找到我的GPU。运行时
sess = tf.Session(config=tf.ConfigProto(log_device_placement=True))
它给出了:
设备映射:没有已知设备。
我遵循了本教程https://nitishmutha.github.io/tensorflow/2017/01/22/TensorFlow-with-gpu-for-windows.html 一步一步,除了Anaconda,因为我用pip3安装了tensorflow。它对我不起作用。
更新
我知道为什么它不起作用。
对于sime的原因,我的python正在使用仅限CPU的tensorflow版本,虽然我有两者。我卸载了CPU版本并再次安装了tensorflow-gpu。现在它给出了错误:
没有名为'_pywrap_tensorflow_internal'的模块
我知道这与安装有关,还有更多的讨论如下:
On Windows, running "import tensorflow" generates No module named "_pywrap_tensorflow" error
Cannot import Tensorflow for GPU on Windows 10
继续努力。