我的keras项目中发生了一个非常陌生的过拟合现象。合适的代码是:
model.fit(
x = trainX,
y = trainY,
batch_size=8,
epochs=30,
validation_split=0.2,
shuffle=True,
callbacks=[checkpoint, tensorboard]
)
打印的信息是:
4184/4320 [============================>.] - ETA: 0s - loss: 0.0481 - accuracy: 0.9804
4208/4320 [============================>.] - ETA: 0s - loss: 0.0499 - accuracy: 0.9800
4232/4320 [============================>.] - ETA: 0s - loss: 0.0496 - accuracy: 0.9802
4256/4320 [============================>.] - ETA: 0s - loss: 0.0494 - accuracy: 0.9803
4280/4320 [============================>.] - ETA: 0s - loss: 0.0491 - accuracy: 0.9804
4312/4320 [============================>.] - ETA: 0s - loss: 0.0489 - accuracy: 0.9805
4320/4320 [==============================] - 10s 2ms/step - loss: 0.0489 - accuracy: 0.9806 - val_loss: 7.5471 - val_accuracy: 0.5093
训练的准确性为0.98,而验证的准确性为0.50。该项目是2类分类项目,因此0.50表示该模型无效。
为什么这种过度拟合会发生?训练数据和验证数据均来自(trainX,trainY)。另外,在拟合之前,我使trainX的顺序随机:
n = trainX.shape[0]
ind = np.arange(n)
np.random.shuffle(ind)
trainX = trainX[ind, :]
trainY = trainY[ind, :]
在我的模型中,我使用了keras.layers.Dropout。我不确定如何解决此问题。