Question

我训练了一个基于4 keras的lstm层，其中包裹了辍学层堆叠模型以进行时间序列预测。结果不是太差，代码是：

regressor = keras.models.Sequential()

# input layer
regressor.add(keras.layers.LSTM(units=50, return_sequences=True, input_shape=(X_train.shape[1], 1)))
regressor.add(keras.layers.Dropout(0.2))

# second layer
regressor.add(keras.layers.LSTM(units=50, return_sequences=True))
regressor.add(keras.layers.Dropout(0.2))

# third layer
regressor.add(keras.layers.LSTM(units=50, return_sequences=True))
regressor.add(keras.layers.Dropout(0.2))

# forth layer
regressor.add(keras.layers.LSTM(units=50))
regressor.add(keras.layers.Dropout(0.2))

# output layer
regressor.add(keras.layers.Dense(units=1))

regressor.compile(optimizer='adam', loss='mean_squared_error')

regressor.fit(X_train, y_train, epochs=1, batch_size=32)

结果图为：

Expected Result(Sorry for the link cause my reputation)

比我想将基于keras的代码转换为纯tensorflow代码。

X = tf.placeholder(tf.float32, [None, num_time_steps, num_inputs])
y = tf.placeholder(tf.float32, [None, predict_time_steps, num_outputs])

cell1 = tf.nn.rnn_cell.BasicLSTMCell(num_units=num_units, activation=tf.nn.relu)
dropout1 = tf.nn.rnn_cell.DropoutWrapper(cell1, input_keep_prob=0.8)

cell2 = tf.nn.rnn_cell.BasicLSTMCell(num_units=num_units, activation=tf.nn.relu)
dropout2 = tf.nn.rnn_cell.DropoutWrapper(cell2, input_keep_prob=0.8)

cell3 = tf.nn.rnn_cell.BasicLSTMCell(num_units=num_units, activation=tf.nn.relu)
dropout3 = tf.nn.rnn_cell.DropoutWrapper(cell3, input_keep_prob=0.8)

cell4 = tf.nn.rnn_cell.BasicLSTMCell(num_units=num_units, activation=tf.nn.relu)
dropout4 = tf.nn.rnn_cell.DropoutWrapper(cell4, input_keep_prob=0.8)

lstm_layers = tf.contrib.rnn.OutputProjectionWrapper(
    tf.nn.rnn_cell.MultiRNNCell([dropout1, dropout2, dropout3, dropout4]), output_size=num_outputs)

outputs, states = tf.nn.dynamic_rnn(lstm_layers, X, dtype=tf.float32)

loss = tf.reduce_mean(tf.square(outputs - y))
optimizer = tf.train.AdamOptimizer(learning_rate=learning_rate)

train = optimizer.minimize(loss)

但是结果是完全错误的。 Wrong Result(Sorry for the link cause my reputation)

出什么问题了？谢谢你们。

Answer 1

我建议您查看文档，例如使用Keras时，您具有以下默认设置：

keras.layers.LSTM（units，activation ='tanh'， recurrent_activation ='hard_sigmoid'，use_bias = True， kernel_initializer ='glorot_uniform'， recurrent_initializer ='正交'，bias_initializer ='零'， unit_forget_bias = True，kernel_regularizer =无， recurrent_regularizer =无，bias_regularizer =无， activity_regularizer = None，kernel_constraint = None， recurrent_constraint =无，bias_constraint =无，辍学= 0.0， recurrent_dropout = 0.0，实现= 1，return_sequences = False， return_state = False，go_backwards = False，有状态= False，展开= False）

TensorFlow这个：

初始化（ num_units，忘了偏置= 1.0， state_is_tuple =真实，激活=无，重用=无，名称=无， dtype = None， ** kwargs）

您能发现差异吗？

为什么基于Keras的多个LSTM模型的性能要比基于纯Tensorflow的时间序列预测好得多？

1 个答案: