我正在处理文本文档的多类分类任务。
我尝试了两种DL模型，即CNN和LSTM。
LSTM模型在Keras输出上显示了较高的acc得分（> 0.91），但是在我计算了预测的准确性（model.predict的最大概率）之后，
acc分数约为0.5。
我在CNN模型上测试了相同的东西，但是acc与Keras输出相同（火车acc 0.96，测试acc 0.71）。因此，我认为我的LSTM代码可能有问题。（它属于link的一部分）
为什么它们不同？
这是Keras输出和LSTM的最大概率精度。
C:\ProgramData\Anaconda3\envs\tensorflow\lib\site-packages\tensorflow\python\ops\gradients_impl.py:112: UserWarning: Converting sparse
IndexedSlices to a dense Tensor of unknown shape. This may consume a large amount of memory.
"Converting sparse IndexedSlices to a dense Tensor of unknown shape. "
Train on 1039 samples, validate on 116 samples
Epoch 1/20
1039/1039 [==============================] - 11s 11ms/step - loss: 0.2707 - acc: 0.9167 - val_loss: 0.2443 - val_acc: 0.9167
Epoch 2/20
1039/1039 [==============================] - 10s 9ms/step - loss: 0.2227 - acc: 0.9179 - val_loss: 0.1944 - val_acc: 0.9217
Epoch 3/20
1039/1039 [==============================] - 10s 9ms/step - loss: 0.1894 - acc: 0.9250 - val_loss: 0.1700 - val_acc: 0.9368
Epoch 4/20
1039/1039 [==============================] - 10s 9ms/step - loss: 0.1694 - acc: 0.9345 - val_loss: 0.1509 - val_acc: 0.9418
Epoch 5/20
1039/1039 [==============================] - 10s 9ms/step - loss: 0.1541 - acc: 0.9394 - val_loss: 0.1405 - val_acc: 0.9432
Epoch 6/20
1039/1039 [==============================] - 10s 9ms/step - loss: 0.1445 - acc: 0.9421 - val_loss: 0.1347 - val_acc: 0.9440
Epoch 7/20
1039/1039 [==============================] - 9s 9ms/step - loss: 0.1360 - acc: 0.9439 - val_loss: 0.1240 - val_acc: 0.9533
Epoch 8/20
1039/1039 [==============================] - 10s 9ms/step - loss: 0.1226 - acc: 0.9520 - val_loss: 0.1200 - val_acc: 0.9555
Epoch 9/20
1039/1039 [==============================] - 10s 9ms/step - loss: 0.1163 - acc: 0.9541 - val_loss: 0.1164 - val_acc: 0.9511
Epoch 10/20
1039/1039 [==============================] - 10s 9ms/step - loss: 0.1088 - acc: 0.9584 - val_loss: 0.1135 - val_acc: 0.9526
Epoch 11/20
1039/1039 [==============================] - 10s 9ms/step - loss: 0.1021 - acc: 0.9609 - val_loss: 0.1070 - val_acc: 0.9612
Epoch 12/20
1039/1039 [==============================] - 10s 9ms/step - loss: 0.0956 - acc: 0.9633 - val_loss: 0.1061 - val_acc: 0.9562
Epoch 13/20
1039/1039 [==============================] - 10s 9ms/step - loss: 0.0911 - acc: 0.9650 - val_loss: 0.1100 - val_acc: 0.9533
Epoch 14/20
1039/1039 [==============================] - 10s 9ms/step - loss: 0.0860 - acc: 0.9680 - val_loss: 0.1003 - val_acc: 0.9641
Epoch 15/20
1039/1039 [==============================] - 10s 9ms/step - loss: 0.0782 - acc: 0.9716 - val_loss: 0.0965 - val_acc: 0.9634
Epoch 16/20
1039/1039 [==============================] - 10s 9ms/step - loss: 0.0716 - acc: 0.9732 - val_loss: 0.0957 - val_acc: 0.9655
Epoch 17/20
1039/1039 [==============================] - 10s 9ms/step - loss: 0.0662 - acc: 0.9752 - val_loss: 0.0984 - val_acc: 0.9648
Epoch 18/20
1039/1039 [==============================] - 9s 9ms/step - loss: 0.0644 - acc: 0.9764 - val_loss: 0.0996 - val_acc: 0.9662
Epoch 19/20
1039/1039 [==============================] - 10s 9ms/step - loss: 0.0614 - acc: 0.9765 - val_loss: 0.0897 - val_acc: 0.9677
Epoch 20/20
1039/1039 [==============================] - 9s 9ms/step - loss: 0.0564 - acc: 0.9780 - val_loss: 0.1023 - val_acc: 0.9713
289/289 [==============================] - 1s 3ms/step
NN Test score: 0.11336773380421257
NN Test accuracy: 0.959919253022613
Train set evaluation:
1155/1155 [==============================] - 4s 3ms/step
C:\ProgramData\Anaconda3\envs\tensorflow\lib\site-packages\sklearn\metrics\classification.py:1135: UndefinedMetricWarning: Precision is ill-defined and being set to 0.0 in labels with no predicted samples.
'precision', 'predicted', average, warn_for)
C:\ProgramData\Anaconda3\envs\tensorflow\lib\site-packages\sklearn\metrics\classification.py:1135: UndefinedMetricWarning: F-score is ill-defined and being set to 0.0 in labels with no predicted samples.
'precision', 'predicted', average, warn_for)
accuracy: 0.5272727272727272
macro precision: 0.36873557666657714
macro recall: 0.3997861468919986
macro f1: 0.361281520651822
micro precision: 0.5272727272727272
micro recall: 0.5272727272727272
micro f1: 0.5272727272727272
[[ 86 30 9 0 7 0 0 0 0 0 0 0]
[ 26 56 1 0 25 0 0 0 0 1 0 0]
[ 1 0 151 0 4 3 1 0 0 0 0 6]
[ 29 0 0 0 16 0 0 0 0 0 0 3]
[ 9 8 0 0 53 0 0 0 0 0 0 60]
[ 0 0 27 0 10 46 0 0 0 1 0 1]
[ 13 1 2 0 0 0 103 0 0 0 0 0]
[ 13 10 3 0 4 0 2 0 0 0 0 0]
[ 17 0 11 0 4 0 0 0 0 6 0 2]
[ 76 12 3 0 17 2 0 0 0 4 0 0]
[ 21 0 1 0 1 0 31 0 0 0 0 0]
[ 0 1 2 0 13 0 0 0 0 0 0 110]]
Test set evaluation:
289/289 [==============================] - 1s 3ms/step
accuracy: 0.5017301038062284
macro precision: 0.33315831055769135
macro recall: 0.3890209913941563
macro f1: 0.351037197057408
micro precision: 0.5017301038062284
micro recall: 0.5017301038062284
micro f1: 0.5017301038062284
[[20 7 3 0 5 0 0 0 0 2 0 0]
[11 15 0 0 7 0 0 0 0 1 0 0]
[ 0 1 28 0 5 2 1 0 0 1 0 0]
[ 5 1 0 0 4 0 0 0 0 0 0 5]
[ 1 3 0 0 18 0 0 0 0 0 0 9]
[ 0 0 5 0 2 12 0 0 0 2 0 0]
[ 0 0 1 0 0 0 27 0 0 0 0 0]
[ 3 5 1 0 0 0 1 0 0 0 0 0]
[ 2 0 2 0 2 0 0 0 0 1 0 0]
[15 3 0 0 3 0 0 0 0 0 0 0]
[ 8 0 0 0 0 0 9 0 0 0 0 0]
[ 0 0 0 0 5 0 0 0 0 0 0 25]]
和代码
def build_model(embedding_weights, maxlen, nb_classes):
n_embedding_vocab = embedding_weights.shape[0]
embedding_dim = embedding_weights.shape[1]
inp = Input(shape=(maxlen,))
x = Embedding(n_embedding_vocab, embedding_dim, weights=[embedding_weights])(inp)
x = Bidirectional(LSTM(50, return_sequences=True, dropout=0.1, recurrent_dropout=0.1))(x)
x = GlobalMaxPool1D()(x)
x = Dense(50, activation="relu")(x)
x = Dropout(0.1)(x)
x = Dense(nb_classes, activation="softmax")(x)
model = Model(inputs=inp, outputs=x)
model.compile(loss='binary_crossentropy', optimizer='adam', metrics=['accuracy'])
return model
def main():
print('X_train:', X_train.shape, 'Y_train:', Y_train.shape)
model = build_model(embedding_weight, maxlen, n_class)
model.fit(X_train, Y_train, batch_size=32, epochs=20, validation_split=0.1)
score = model.evaluate(X_test, Y_test, batch_size=32, verbose=1)
print('NN Test score:', score[0])
print('NN Test accuracy:', score[1])
print('Train set evaluation:')
pred_probs = model.predict([X_train], batch_size=32, verbose=1)
model.save(self.lstm.model_path)
pred = np.argmax(pred_probs, axis=1)
getEvaluate_SLC(targets[train_idx], pred, label_encoder)
print('Test set evaluation:')
pred_probs = model.predict([X_test], batch_size=32, verbose=1)
model.save(self.lstm.model_path)
pred = np.argmax(pred_probs, axis=1)
getEvaluate_SLC(targets[test_idx], pred, label_encoder)