过度拟合数据集的可能原因

时间:2017-03-28 18:17:46

标签: tensorflow neural-network keras

我使用的数据集包含33k图像。培训包含27k,验证集包含6k图像 我使用以下CNN代码作为模型:

model = Sequential()

model.add(Convolution2D(32, 3, 3, activation='relu', border_mode="same", input_shape=(row, col, ch)))
model.add(Convolution2D(32, 3, 3, activation='relu', border_mode="same"))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Dropout(0.25))
model.add(Convolution2D(64, 3, 3, activation='relu', border_mode="same"))
model.add(Convolution2D(128, 3, 3, activation='relu', border_mode="same"))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Dropout(0.25))
model.add(Flatten())
model.add(Activation('relu'))
model.add(Dense(1024))
model.add(Dropout(0.5))
model.add(Activation('relu'))
model.add(Dense(1))
adam = Adam(lr=0.0001)
model.compile(optimizer=adam, loss="mse", metrics=["mae"])

我获得的输出有decreasing training lossincreasing validation loss建议overfitting。但是我已经在dropouts中加入了preventing overfitting这应该有所帮助。以下是经过10个时期训练时输出的快照:

Epoch 1/10
27008/27040 [============================>.] - ETA: 5s - loss: 0.0629 - mean_absolute_error: 0.1428 Epoch 00000: val_loss improved from inf to 0.07595, saving model to dataset/-00-val_loss_with_mymodel-0.08.hdf5
27040/27040 [==============================] - 4666s - loss: 0.0629 - mean_absolute_error: 0.1428 - val_loss: 0.0759 - val_mean_absolute_error: 0.1925
Epoch 2/10
27008/27040 [============================>.] - ETA: 5s - loss: 0.0495 - mean_absolute_error: 0.1287 Epoch 00001: val_loss did not improve
27040/27040 [==============================] - 4605s - loss: 0.0494 - mean_absolute_error: 0.1287 - val_loss: 0.0946 - val_mean_absolute_error: 0.2289
Epoch 3/10
27008/27040 [============================>.] - ETA: 5s - loss: 0.0382 - mean_absolute_error: 0.1119 Epoch 00002: val_loss did not improve
27040/27040 [==============================] - 4610s - loss: 0.0382 - mean_absolute_error: 0.1119 - val_loss: 0.1081 - val_mean_absolute_error: 0.2463

So, what is wrong? Are there any other methods to prevent overfitting?
Does shuffling of data help?

1 个答案:

答案 0 :(得分:1)

我会尝试添加1E-4的重量衰减。这可以通过添加权重衰减层来完成,如下所示:model.add(Convolution2D(32, 3, 3, activation='relu', border_mode="same", input_shape=(row, col, ch), W_regularizer=l2(1E-4), b_regularizer=l2(1E-4)))。 L2可以在keras.regularizershttps://keras.io/regularizers/#example)中找到。重量正则化非常适合对抗过度拟合。

然而,过度拟合可能不仅是您的模型的结果,也是您的模型的结果。如果验证数据在某种程度上“更难”,那么您的列车数据则可能只是您不能适应它。