Question

我有一个非常简单的用Keras编写的LSTM示例，我正尝试移植到pytorch。但是它似乎根本无法学习。我是一个绝对的开始，因此任何建议都值得赞赏。

KERAS ：

X_train_lmse的形状为(1691, 1, 1)，实际上我在以X(t)作为单一功能运行X(t-1)

lstm_model = Sequential()
lstm_model.add(LSTM(7, input_shape=(1, X_train_lmse.shape[1]), activation='relu', kernel_initializer='lecun_uniform', return_sequences=False))
lstm_model.add(Dense(1))
lstm_model.compile(loss='mean_squared_error', optimizer='adam')
early_stop = EarlyStopping(monitor='loss', patience=2, verbose=1)
history_lstm_model = lstm_model.fit(X_train_lmse, y_train, epochs=100, batch_size=1, verbose=1, shuffle=False, callbacks=[early_stop])

输出：

Epoch 1/100
1691/1691 [==============================] - 10s 6ms/step - loss: 0.0236
Epoch 2/100
1691/1691 [==============================] - 9s 5ms/step - loss: 0.0076
Epoch 3/100
...

PYTORCH ：

X_train_tensor具有与keras（1691，1，1）中相同的形状。我在下面指定batch_first为true，所以我认为应该没问题。

class LSTM_model(nn.Module):

    def __init__(self):
        super(LSTM_model, self).__init__()
        self.lstm = nn.LSTM(input_size=1, hidden_size=7, num_layers=1, batch_first=True)
        self.dense = nn.Linear(7, 1)

    def forward(self, x):
        out, states = self.lstm(x)
        out = self.dense(out)
        return out

lstm_model = LSTM_model()
loss_function = nn.MSELoss()
optimizer = optim.Adam(lstm_model.parameters())

for t in range(100):
    y_pred = lstm_model(X_train_tensor)
    loss = loss_function(y_pred, Y_train_tensor)
    optimizer.zero_grad()
    loss.backward()
    optimizer.step()
    print('Train Epoch ', t, ' Loss = ', loss)

输出：

Train Epoch  0  Loss =  tensor(0.2834, grad_fn=<MseLossBackward>)
Train Epoch  1  Loss =  tensor(0.2812, grad_fn=<MseLossBackward>)
Train Epoch  2  Loss =  tensor(0.2790, grad_fn=<MseLossBackward>)
Train Epoch  3  Loss =  tensor(0.2768, grad_fn=<MseLossBackward>)
Train Epoch  4  Loss =  tensor(0.2746, grad_fn=<MseLossBackward>)
Train Epoch  5  Loss =  tensor(0.2725, grad_fn=<MseLossBackward>)
Train Epoch  6  Loss =  tensor(0.2704, grad_fn=<MseLossBackward>)
Train Epoch  7  Loss =  tensor(0.2683, grad_fn=<MseLossBackward>)
...

如您所见，该错误在Pytorch中几乎没有移动。而且每个纪元的运行都比喀拉拉邦快得多。

我一定在做愚蠢的事。我检查了输入数据，在两种实现中它们看起来都相同。谢谢！

Answer 1

您错过了PyTorch模型（See Relu layer in PyTorch）中的relu激活功能。另外，您似乎正在使用自定义的kernel_initalizer作为权重。您可以在模型调用中传递初始化权重：

...
y_pred = lstm_model(X_train_tensor, (hn, cn))
...

怎么把keras LSTM转换成pytorch LSTM？

1 个答案: