Question

我正在尝试建立模型来预测文本。

x_train的形状为（19992，40，1）

array([[[0.00680272],
        [0.01417234],
        [0.        ],
        ...,

        [0.01473923],
        [0.        ],
        [0.0085034 ]]])

y_train的形状为（19992，42）（它是一热编码的）

array([[0., 0., 0., ..., 0., 0., 0.],
       [0., 0., 0., ..., 0., 0., 0.],
       [0., 0., 0., ..., 0., 0., 0.],
       ...,
       [0., 0., 0., ..., 0., 0., 0.],
       [1., 0., 0., ..., 0., 0., 0.],
       [0., 0., 0., ..., 0., 0., 0.]], dtype=float32)

我的模特是：

#####################################################################################################
model = Sequential()
model.add(LSTM(256,input_shape=(40,1),return_sequences=True))
model.add(Dropout(0.3))
model.add(LSTM(128))
model.add(Dropout(0.5))
model.add(Dense(42,activation='softmax'))

model.compile(optimizer='RMSprop',loss='categorical_crossentropy',metrics=['accuracy'])

现在即使以150个历元来训练我的模型，我也只能达到0.512的精度。我应该在模型中进行哪些改进以提高其准确性？

Train on 15993 samples, validate on 3999 samples
Epoch 1/15
15993/15993 [==============================] - 23s 3ms/step - loss: 2.9527 - acc: 0.2013 - val_loss: 2.8762 - val_acc: 0.2061
Epoch 2/15
15993/15993 [==============================] - 23s 3ms/step - loss: 2.8670 - acc: 0.2111 - val_loss: 2.8678 - val_acc: 0.2061
Epoch 3/15
15993/15993 [==============================] - 23s 3ms/step - loss: 2.8548 - acc: 0.2117 - val_loss: 2.8615 - val_acc: 0.2061
Epoch 4/15
15993/15993 [==============================] - 22s 3ms/step - loss: 2.8516 - acc: 0.2121 - val_loss: 2.8629 - val_acc: 0.2061
Epoch 5/15
15993/15993 [==============================] - 22s 3ms/step - loss: 2.8447 - acc: 0.2117 - val_loss: 2.8663 - val_acc: 0.2061
Epoch 6/15
15993/15993 [==============================] - 21s 3ms/step - loss: 2.8445 - acc: 0.2133 - val_loss: 2.8657 - val_acc: 0.2061
Epoch 7/15
15993/15993 [==============================] - 22s 3ms/step - loss: 2.8404 - acc: 0.2134 - val_loss: 2.8657 - val_acc: 0.2061
Epoch 8/15
15993/15993 [==============================] - 21s 3ms/step - loss: 2.8401 - acc: 0.2117 - val_loss: 2.8673 - val_acc: 0.2061
Epoch 9/15
15993/15993 [==============================] - 21s 3ms/step - loss: 2.8391 - acc: 0.2139 - val_loss: 2.8657 - val_acc: 0.2061
Epoch 10/15
15993/15993 [==============================] - 22s 3ms/step - loss: 2.8412 - acc: 0.2141 - val_loss: 2.8642 - val_acc: 0.2061
Epoch 11/15
15993/15993 [==============================] - 21s 3ms/step - loss: 2.8394 - acc: 0.2149 - val_loss: 2.8680 - val_acc: 0.2061
Epoch 12/15
15993/15993 [==============================] - 22s 3ms/step - loss: 2.8404 - acc: 0.2154 - val_loss: 2.8658 - val_acc: 0.2061
Epoch 13/15
15993/15993 [==============================] - 22s 3ms/step - loss: 2.8380 - acc: 0.2161 - val_loss: 2.8672 - val_acc: 0.2061
Epoch 14/15
15993/15993 [==============================] - 22s 3ms/step - loss: 2.8384 - acc: 0.2169 - val_loss: 2.8674 - val_acc: 0.2061
Epoch 15/15
15993/15993 [==============================] - 22s 3ms/step - loss: 2.8378 - acc: 0.2171 - val_loss: 2.8702 - val_acc: 0.2061

Answer 1

我认为您正在考虑基于LSTM的字符级语言模型。这种模型通常使用多维嵌入作为输入，而不仅仅是一维标量。因此，对于Keras，您可以尝试以下网络体系结构：

model = Sequential()
model.add(Embedding(42, output_dim=64, input_length=40))
model.add(LSTM(256,input_shape=(40,1),return_sequences=True))
model.add(Dropout(0.3))
model.add(LSTM(128))
model.add(Dropout(0.5))
model.add(Dense(42,activation='softmax'))

其中output_dim是嵌入尺寸的数量。该网络的输入是整数矩阵[batch_size x input_length]，其中每个元素都是char索引。详细了解this post。希望这会有所帮助！

LSTM模型的准确性非常低

1 个答案: