Question

我建立了一个LSTM模型，该模型在我的时间序列问题上表现很好。我现在想做的是结合辍学和经常性辍学，对我的预测的不确定性进行建模。从我读过的文献中，这应该会给我的预测增加噪音，但是在推断期间不使用掉落的情况下，这些预测的平均值不应与值相差太远。就我而言，当使用辍学时，即使只有0.01，我的预测精度也比没有辍学时高约一个数量级。例如，如果我评估一个批次（16、20、14），而没有辍学，结果是相同的。

for x, y in test.take(1):
  for i in range(10):
    encoder_decoder_dropout.evaluate(x, y)

1/1 [==============================] - 0s 2ms/step - loss: 0.0584 - mean_absolute_error: 0.0584
1/1 [==============================] - 0s 1ms/step - loss: 0.0584 - mean_absolute_error: 0.0584
1/1 [==============================] - 0s 1ms/step - loss: 0.0584 - mean_absolute_error: 0.0584
1/1 [==============================] - 0s 1ms/step - loss: 0.0584 - mean_absolute_error: 0.0584
1/1 [==============================] - 0s 2ms/step - loss: 0.0584 - mean_absolute_error: 0.0584
1/1 [==============================] - 0s 1ms/step - loss: 0.0584 - mean_absolute_error: 0.0584
1/1 [==============================] - 0s 1ms/step - loss: 0.0584 - mean_absolute_error: 0.0584
1/1 [==============================] - 0s 1ms/step - loss: 0.0584 - mean_absolute_error: 0.0584
1/1 [==============================] - 0s 1ms/step - loss: 0.0584 - mean_absolute_error: 0.0584
1/1 [==============================] - 0s 1ms/step - loss: 0.0584 - mean_absolute_error: 0.0584

但是，添加0.1的差值后，我看到了结果的随机性，但是准确性很差：

for x, y in test.take(1):
  for i in range(10):
    encoder_decoder_dropout.evaluate(x, y)

1/1 [==============================] - 0s 2ms/step - loss: 0.5677 - mean_absolute_error: 0.5677
1/1 [==============================] - 0s 1ms/step - loss: 0.5512 - mean_absolute_error: 0.5512
1/1 [==============================] - 0s 1ms/step - loss: 0.6085 - mean_absolute_error: 0.6085
1/1 [==============================] - 0s 1ms/step - loss: 0.6091 - mean_absolute_error: 0.6091
1/1 [==============================] - 0s 1ms/step - loss: 0.6503 - mean_absolute_error: 0.6503
1/1 [==============================] - 0s 1ms/step - loss: 0.5438 - mean_absolute_error: 0.5438
1/1 [==============================] - 0s 1ms/step - loss: 0.5882 - mean_absolute_error: 0.5882
1/1 [==============================] - 0s 1ms/step - loss: 0.6500 - mean_absolute_error: 0.6500
1/1 [==============================] - 0s 1ms/step - loss: 0.6235 - mean_absolute_error: 0.6235
1/1 [==============================] - 0s 1ms/step - loss: 0.6119 - mean_absolute_error: 0.6119

我的模特在下面。我想知道这是否是预期的行为，还是我对MCdropout的实现不佳。

input= keras.Input(shape=(60))
x = layers.RepeatVector(30, input_shape=[60])(input)
x = layers.Bidirectional(tf.keras.layers.LSTM(100, dropout=0.1, 
                                                    recurrent_dropout=0.1,
                                                     return_sequences=False))(x, training=True)
x = layers.Dense(look_forward*num_features,
                        kernel_initializer=tf.initializers.glorot_normal())(x)
output = layers.Reshape([look_forward, num_features])(x)

decoder_dropout = keras.Model(input, output, name="decoder_dropout")

encoder_decoder_dropout = tf.keras.Sequential([encoder, decoder_dropout])
encoder_decoder_dropout.set_weights(encoder_decoder_dropout_trained.get_weights()) 

encoder_decoder_dropout.compile(loss=tf.keras.losses.MeanAbsoluteError(),
                  optimizer=tf.optimizers.Adam(),
                  metrics=[tf.metrics.MeanAbsoluteError()])

正确使用LSTM的辍学和不确定性估计

0 个答案: