Question

在回顾有关多维输入和堆叠LSTM RNN的众多类似问题时，我没有找到一个示例，该示例列出了initial_state占位符并遵循以下rnn_tuple_state的维数。尝试的[lstm_num_layers, 2, None, lstm_num_cells, 2]是这些示例（http://monik.in/a-noobs-guide-to-implementing-rnn-lstm-using-tensorflow/，https://medium.com/@erikhallstrm/using-the-tensorflow-multilayered-lstm-api-f6e7da7bbe40）中代码的扩展，并在末尾添加了feature_dim的额外维度，每次都针对多个值功能的步骤（这不起作用，但是由于ValueError调用中的尺寸不匹配而产生了tensorflow.nn.dynamic_rnn）。

time_steps = 10
feature_dim = 2
label_dim = 4
lstm_num_layers = 3
lstm_num_cells = 100
dropout_rate = 0.8

# None is to allow for variable size batches
features = tensorflow.placeholder(tensorflow.float32,
                                  [None, time_steps, feature_dim])
labels = tensorflow.placeholder(tensorflow.float32, [None, label_dim])

cell = tensorflow.contrib.rnn.MultiRNNCell(
    [tensorflow.contrib.rnn.LayerNormBasicLSTMCell(
        lstm_num_cells,
        dropout_keep_prob = dropout_rate)] * lstm_num_layers,
    state_is_tuple = True)

# not sure of the dimensionality for the initial state
initial_state = tensorflow.placeholder(
    tensorflow.float32,
    [lstm_num_layers, 2, None, lstm_num_cells, feature_dim])
# which impacts these two lines as well
state_per_layer_list = tensorflow.unstack(initial_state, axis = 0)
rnn_tuple_state = tuple(
    [tensorflow.contrib.rnn.LSTMStateTuple(
        state_per_layer_list[i][0],
        state_per_layer_list[i][1]) for i in range(lstm_num_layers)])

# also not sure if expanding the feature dimensions is correct here
outputs, state = tensorflow.nn.dynamic_rnn(
    cell, tensorflow.expand_dims(features, -1),
    initial_state = rnn_tuple_state)

最有用的是对以下情况的解释：

每个时间步长具有 N 个值
每个时间序列具有 S 个步骤
每批具有 B 序列
每个输出具有 R 个值
网络中存在 L 个隐藏的LSTM层
每个层的节点数为 M

所以它的伪代码版本是：

# B, S, N, and R are undefined values for the purpose of this question
features = tensorflow.placeholder(tensorflow.float32, [B, S, N])
labels = tensorflow.placeholder(tensorflow.float32, [B, R])
...

如果我可以完成的话，我不会首先在这里问。提前致谢。欢迎您对有关最佳做法的任何评论。

Answer 1

经过反复试验，无论特征的尺寸如何，以下代码都会生成堆叠的LSTM dynamic_rnn：

time_steps = 10
feature_dim = 2
label_dim = 4
lstm_num_layers = 3
lstm_num_cells = 100
dropout_rate = 0.8
learning_rate = 0.001

features = tensorflow.placeholder(
    tensorflow.float32, [None, time_steps, feature_dim])
labels = tensorflow.placeholder(
    tensorflow.float32, [None, label_dim])

cell_list = []
for _ in range(lstm_num_layers):
    cell_list.append(
        tensorflow.contrib.rnn.LayerNormBasicLSTMCell(lstm_num_cells,
                                                      dropout_keep_prob=dropout_rate))
cell = tensorflow.contrib.rnn.MultiRNNCell(cell_list, state_is_tuple=True)
initial_state = tensorflow.placeholder(
    tensorflow.float32, [lstm_num_layers, 2, None, lstm_num_cells])
state_per_layer_list = tensorflow.unstack(initial_state, axis=0)
rnn_tuple_state = tuple(
    [tensorflow.contrib.rnn.LSTMStateTuple(
        state_per_layer_list[i][0],
        state_per_layer_list[i][1]) for i in range(lstm_num_layers)])
state_series, last_state = tensorflow.nn.dynamic_rnn(
    cell=cell, inputs=features, initial_state=rnn_tuple_state)

hidden_layer_output = tensorflow.transpose(state_series, [1, 0, 2])
last_output = tensorflow.gather(hidden_layer_output, int(
    hidden_layer_output.get_shape()[0]) - 1)

weights = tensorflow.Variable(tensorflow.random_normal(
    [lstm_num_cells, int(labels.get_shape()[1])]))
biases = tensorflow.Variable(tensorflow.constant(
    0.0, shape=[labels.get_shape()[1]]))
predictions = tensorflow.matmul(last_output, weights) + biases
mean_squared_error = tensorflow.reduce_mean(
    tensorflow.square(predictions - labels))
minimize_error = tensorflow.train.RMSPropOptimizer(
    learning_rate).minimize(mean_squared_error)

从许多众所周知的兔子洞之一开始此旅程的部分原因是，先前引用的示例对输出进行了重塑，以容纳分类器而不是回归器（这是我试图构建的）。由于这与特征维数无关，因此可以用作该用例的通用模板。

TensorFlow中堆叠LSTM网络的尺寸

1 个答案: