验证准确性停滞不前，而培训准确性却有所提高

Question

我对深度学习还很陌生，所以如果我遗漏了一些明显的东西，我感到抱歉。

我目前正在使用整合在一起的数据集训练CNN。 训练时，训练准确性表现得很正常，并且会提高，达到> 99％的准确性。我的验证准确性开始时约为75％，并在81％±1％左右波动。经过训练后，该模型在全新数据上的表现非常好。

Epoch 1/100
187/187 [==============================] - 103s 550ms/step - loss: 1.1336 - acc: 0.5384 - val_loss: 0.8065 - val_acc: 0.7405
Epoch 2/100
187/187 [==============================] - 97s 519ms/step - loss: 0.8041 - acc: 0.7345 - val_loss: 0.7566 - val_acc: 0.7720
Epoch 3/100
187/187 [==============================] - 97s 519ms/step - loss: 0.7194 - acc: 0.7945 - val_loss: 0.7410 - val_acc: 0.7846
Epoch 4/100
187/187 [==============================] - 97s 517ms/step - loss: 0.6688 - acc: 0.8324 - val_loss: 0.7295 - val_acc: 0.7924
Epoch 5/100
187/187 [==============================] - 97s 518ms/step - loss: 0.6288 - acc: 0.8611 - val_loss: 0.7197 - val_acc: 0.7961
Epoch 6/100
187/187 [==============================] - 96s 515ms/step - loss: 0.5989 - acc: 0.8862 - val_loss: 0.7252 - val_acc: 0.7961
Epoch 7/100
187/187 [==============================] - 96s 514ms/step - loss: 0.5762 - acc: 0.8981 - val_loss: 0.7135 - val_acc: 0.8063
Epoch 8/100
187/187 [==============================] - 97s 518ms/step - loss: 0.5513 - acc: 0.9186 - val_loss: 0.7089 - val_acc: 0.8077
Epoch 9/100
187/187 [==============================] - 96s 513ms/step - loss: 0.5351 - acc: 0.9280 - val_loss: 0.7113 - val_acc: 0.8053
Epoch 10/100
187/187 [==============================] - 96s 514ms/step - loss: 0.5189 - acc: 0.9417 - val_loss: 0.7167 - val_acc: 0.8094
Epoch 11/100
187/187 [==============================] - 96s 515ms/step - loss: 0.5026 - acc: 0.9483 - val_loss: 0.7104 - val_acc: 0.8162
Epoch 12/100
187/187 [==============================] - 96s 516ms/step - loss: 0.4914 - acc: 0.9538 - val_loss: 0.7114 - val_acc: 0.8101
Epoch 13/100
187/187 [==============================] - 96s 515ms/step - loss: 0.4809 - acc: 0.9583 - val_loss: 0.7099 - val_acc: 0.8141
Epoch 14/100
187/187 [==============================] - 96s 512ms/step - loss: 0.4681 - acc: 0.9656 - val_loss: 0.7149 - val_acc: 0.8182
Epoch 15/100
187/187 [==============================] - 96s 515ms/step - loss: 0.4605 - acc: 0.9701 - val_loss: 0.7139 - val_acc: 0.8172
Epoch 16/100
187/187 [==============================] - 96s 514ms/step - loss: 0.4479 - acc: 0.9753 - val_loss: 0.7102 - val_acc: 0.8182
Epoch 17/100
187/187 [==============================] - 96s 513ms/step - loss: 0.4418 - acc: 0.9805 - val_loss: 0.7087 - val_acc: 0.8247
Epoch 18/100
187/187 [==============================] - 96s 512ms/step - loss: 0.4363 - acc: 0.9809 - val_loss: 0.7148 - val_acc: 0.8213
Epoch 19/100
187/187 [==============================] - 96s 516ms/step - loss: 0.4225 - acc: 0.9870 - val_loss: 0.7184 - val_acc: 0.8203
Epoch 20/100
187/187 [==============================] - 96s 513ms/step - loss: 0.4241 - acc: 0.9863 - val_loss: 0.7216 - val_acc: 0.8189
Epoch 21/100
187/187 [==============================] - 96s 513ms/step - loss: 0.4132 - acc: 0.9908 - val_loss: 0.7143 - val_acc: 0.8199
Epoch 22/100
187/187 [==============================] - 96s 515ms/step - loss: 0.4050 - acc: 0.9936 - val_loss: 0.7109 - val_acc: 0.8233
Epoch 23/100
187/187 [==============================] - 96s 515ms/step - loss: 0.4040 - acc: 0.9928 - val_loss: 0.7118 - val_acc: 0.8203
Epoch 24/100
187/187 [==============================] - 96s 511ms/step - loss: 0.3989 - acc: 0.9930 - val_loss: 0.7194 - val_acc: 0.8165
Epoch 25/100
187/187 [==============================] - 97s 517ms/step - loss: 0.3933 - acc: 0.9946 - val_loss: 0.7163 - val_acc: 0.8155
Epoch 26/100
187/187 [==============================] - 97s 516ms/step - loss: 0.3884 - acc: 0.9957 - val_loss: 0.7225 - val_acc: 0.8148
Epoch 27/100
187/187 [==============================] - 95s 510ms/step - loss: 0.3876 - acc: 0.9959 - val_loss: 0.7224 - val_acc: 0.8179

该图本身看起来似乎过度拟合，但是我已经采取了很多措施来解决过度拟合，但似乎没有一个可行。这是我的模型：

# transfer learning with ResNet50
base_model=ResNet50(weights='imagenet', include_top=False, input_shape=(224, 224, 3))

# function to finetune model
def build_finetune_model(base_model, dropout, fc_layers, num_classes):
    # make base model untrainable
    for layer in base_model.layers:
        layer.trainable = False

    x = base_model.output
    x = Flatten()(x)

    # add dense layers
    for fc in fc_layers:
        # use regularizer
        x = Dense(fc, use_bias=False, kernel_regularizer=l2(0.003))(x)
        # add batch normalization
        x = BatchNormalization()(x)
        x = Activation('relu')(x)
        # add dropout
        x = Dropout(dropout)(x)

    # New softmax layer
    x = Dense(num_classes, use_bias=False)(x) 
    x = BatchNormalization()(x)
    predictions = Activation('softmax')(x)

    finetune_model = Model(inputs=base_model.input, outputs=predictions)
    return finetune_model


FC_LAYERS = [1024, 1024]
dropout = 0.5

model = build_finetune_model(base_model, dropout=dropout, fc_layers=FC_LAYERS,num_classes=len(categories))

我正在调整班级权重，并设置了非常低的学习率，希望减慢学习速度。

model.compile(optimizer=Adam(lr=0.000005),loss='categorical_crossentropy',metrics=['accuracy'], weighted_metrics=class_weight)

我对认证准确性开始如此之高（明显高于培训准确性）而在整个培训过程中几乎没有改善这一事实感到非常困惑。如前所述，它似乎过拟合，但是我添加了辍学，批处理规范化和正则化程序，它似乎不起作用。使用水平翻转，随机裁剪，随机亮度和旋转来增强数据也不会显着改变精度。在我的训练数据的ImageDataGenerator().flow_from_directory()中关闭我的数据的随机播放，使得该模型的训练精度大约为25％，而验证精度为<50％（编辑：精度似乎太低了，因为学习率太低了这种情况）。

同样，该模型在新的测试数据上的效果非常好。我正在寻找提高验证准确性的方法，并且想了解神经网络为何以这种方式运行。任何帮助，不胜感激！

Answer 1

您的模型过拟合。您可能要在图像模型上使用数据增强。例如使用ImageDataGenerator（https://keras.io/preprocessing/image/）随机移动，旋转和裁剪图像。

SGD试图找到最简单的方法来最小化数据集上的损失函数；给定足够大的数据点集，就不得不提出通用解决方案；但是DNN尽可能地“记住”输入，因为这是减少损失的最简单方法。辍学和正则化确实有帮助，但是最重要的是验证指标。当然，假设您的验证集已正确平衡。