Question

我正忙着弄清楚我的LSTM模型有什么问题。我有11个输入和2个输出类（一个热编码）并且非常快，就像在1批左右一样，错误只是输出类别之一的百分比并保持不变。

我尝试过印刷重量和偏差，但它们似乎都充满了NaN。

如果我降低了学习率，或者使用图层/单位，我可以慢慢地达到一个类错误的百分比，但似乎总是达到这一点。

以下是代码：

num_units = 30
num_layers = 50
dropout_rate = 0.80
learning_rate=0.0001
batch_size = 180
epoch = 1

input_classes = len(train_input[0])
output_classes = len(train_output[0])

data = tf.placeholder(tf.float32, [None, input_classes, 1]) #Number of examples, number of input, dimension of each input
target = tf.placeholder(tf.float32, [None, output_classes]) #one-hot encoded: [1,0] = bad, [0,1] = good
dropout = tf.placeholder(tf.float32)

cell = tf.contrib.rnn.LSTMCell(num_units, state_is_tuple=True)
cell = tf.contrib.rnn.DropoutWrapper(cell, output_keep_prob=dropout)
cell = tf.contrib.rnn.MultiRNNCell([cell] * num_layers, state_is_tuple=True)

#Input shape [batch_size, max_time, depth], output shape: [batch_size, max_time, cell.output_size]
val, _ = tf.nn.dynamic_rnn(cell, data, dtype=tf.float32) 

val = tf.transpose(val, [1, 0, 2]) #reshapes it to [sequence_size, batch_size, depth]

#get last entry as it includes previous results
last = tf.gather(val, int(val.get_shape()[0]) - 1)

weight = tf.get_variable("W", shape=[num_units, output_classes], initializer=tf.contrib.layers.xavier_initializer())
bias   = tf.get_variable("B", shape=[output_classes], initializer=tf.contrib.layers.xavier_initializer())
logits = tf.matmul(last, weight) + bias

prediction = tf.nn.softmax_cross_entropy_with_logits(logits=logits, labels=target)
prediction = tf.clip_by_value(prediction, 1e-10,100.0)

cost = tf.reduce_mean(prediction)

optimizer = tf.train.AdamOptimizer(learning_rate=learning_rate)
minimize = optimizer.minimize(cost)

mistakes = tf.not_equal(tf.argmax(target, 1), tf.argmax(logits, 1))
error = tf.reduce_mean(tf.cast(mistakes, tf.float32))

init_op = tf.global_variables_initializer()
saver = tf.train.Saver()
sess = tf.Session()
sess.run(init_op)

no_of_batches = int((len(train_input)) / batch_size)
for i in range(epoch):
    ptr = 0
    for j in range(no_of_batches):
        inp, out = train_input[ptr:ptr+batch_size], train_output[ptr:ptr+batch_size]
        ptr+=batch_size
        sess.run(minimize,{data: inp, target: out, dropout: dropout_rate })

sess.close()

Answer 1

由于您有一个热门编码，请使用 sparse_softmax_cross_entropy_with_logits ，而不是 tf.nn.softmax_cross_entropy_with_logits 。

请参阅此stackoverflow答案以了解两个功能的区别 1

LSTM模型错误是一个输出类的百分比

1 个答案: