我正在尝试创建一个用DropoutWrapper和ResidualWrapper包装的LSTM单元格的MultiRNNCell。对于使用variational_recurrent = True,我们必须向DropoutWrapper提供input_size参数。我无法弄清楚应该将input_size传递给每个LSTM层,因为ResidualWrapper还添加了跳过连接以增加每层的输入。
我正在使用以下实用程序功能来创建一个LSTM图层:
def create_cell(units, residual_connections, keep_prob, input_size):
lstm_cell = tf.nn.rnn_cell.LSTMCell(units,
activation=tf.nn.tanh,
initializer=tf.truncated_normal_initializer(),
cell_clip=5.)
lstm_cell = tf.nn.rnn_cell.DropoutWrapper(lstm_cell,
dtype=tf.float32,
input_keep_prob=keep_prob,
output_keep_prob=keep_prob,
state_keep_prob=keep_prob,
variational_recurrent=True,
input_size=input_size)
if residual_connections:
lstm_cell = tf.nn.rnn_cell.ResidualWrapper(lstm_cell)
return lstm_cell
以下代码创建完整的单元格:
net = tf.layers.dense(inputs,
128,
activation=tf.nn.relu,
kernel_initializer=tf.variance_scaling_initializer())
net = tf.layers.batch_normalization(net, training=training)
cells = [create_cell(64, False, keep_prob, ??)]
for _ in range(5):
cells.append(create_cell(64, True, keep_prob, ??))
multirnn_cell = tf.nn.rnn_cell.MultiRNNCell(cells)
net, rnn_s1 = tf.nn.dynamic_rnn(cell=multirnn_cell, inputs=net, initial_state=rnn_s0, dtype=tf.float32)
对于第一个和后续LSTM图层,应该将哪些值传递给input_size?