Question

我正在微调称为C3D的3D-CNN，该3D-CNN最初受过训练，可以根据视频片段对体育运动进行分类。

我正在冻结卷积（特征提取）层，并使用GIPHY的gif训练完全连接的层，以对gif进行分类以进行情感分析（正面或负面）。

除了最终的完全连接层之外，所有层都预先加载了重量。

我正在使用5000张图像（2500幅正片，2500幅负片）进行训练，并使用Keras进行70/30的训练/测试分割。我正在以0.0001的学习率使用Adam优化器。

在训练过程中，训练准确性提高，训练损失减少，但是在验证准确性的早期，随着模型开始过拟合，损失没有改善。

我相信我有足够的训练数据，并且在两个完全连接的层上都使用了0.5的差值，那么如何解决这种过度拟合的问题？

Keras的模型架构，培训代码和培训表现的可视化内容如下。

train_c3d.py

from training.c3d_model import create_c3d_sentiment_model
from ImageSentiment import load_gif_data
import numpy as np
import pathlib
from keras.callbacks import ModelCheckpoint
from keras.optimizers import Adam


def image_generator(files, batch_size):
    """
    Generate batches of images for training instead of loading all images into memory
    :param files:
    :param batch_size:
    :return:
    """
    while True:
        # Select files (paths/indices) for the batch
        batch_paths = np.random.choice(a=files,
                                       size=batch_size)
        batch_input = []
        batch_output = []

        # Read in each input, perform preprocessing and get labels
        for input_path in batch_paths:
            input = load_gif_data(input_path)
            if "pos" in input_path:  # if file name contains pos
                output = np.array([1, 0])  # label
            elif "neg" in input_path:  # if file name contains neg
                output = np.array([0, 1])  # label

            batch_input += [input]
            batch_output += [output]
        # Return a tuple of (input,output) to feed the network
        batch_x = np.array(batch_input)
        batch_y = np.array(batch_output)

        yield (batch_x, batch_y)


model = create_c3d_sentiment_model()
print(model.summary())
model.load_weights('models/C3D_Sport1M_weights_keras_2.2.4.h5', by_name=True)

for layer in model.layers[:14]:  # freeze top layers as feature extractor
    layer.trainable = False
for layer in model.layers[14:]:  # fine tune final layers
    layer.trainable = True

train_files = [str(filepath.absolute()) for filepath in pathlib.Path('data/sample_train').glob('**/*')]
val_files = [str(filepath.absolute()) for filepath in pathlib.Path('data/sample_validation').glob('**/*')]

batch_size = 8
train_generator = image_generator(train_files, batch_size)
validation_generator = image_generator(val_files, batch_size)

model.compile(optimizer=Adam(lr=0.0001),
              loss='binary_crossentropy',
              metrics=['accuracy'])

mc = ModelCheckpoint('best_model.h5', monitor='val_loss', mode='min', verbose=1)

history = model.fit_generator(train_generator, validation_data=validation_generator,
                              steps_per_epoch=int(np.ceil(len(train_files) / batch_size)),
                              validation_steps=int(np.ceil(len(val_files) / batch_size)), epochs=5, shuffle=True,
                              callbacks=[mc])

load_gif_data（）

def load_gif_data(file_path):
    """
    Load and process gif for input into Keras model
    :param file_path:
    :return: Mean normalised image in BGR format as numpy array
             for more info see -> http://cs231n.github.io/neural-networks-2/
    """
    im = Img(fp=file_path)
    try:
        im.load(limit=16,  # Keras image model only requires 16 frames
                first=True)
    except:
        print("Error loading image: " + file_path)
        return
    im.resize(size=(112, 112))
    im.convert('RGB')
    im.close()

    np_frames = []
    frame_index = 0
    for i in range(16):  # if image is less than 16 frames, repeat the frames until there are 16
        frame = im.frames[frame_index]
        rgb = np.array(frame)
        bgr = rgb[..., ::-1]
        mean = np.mean(bgr, axis=0)
        np_frames.append(bgr - mean)  # C3D model was originally trained on BGR, mean normalised images
        # it is important that unseen images are in the same format
        if frame_index == (len(im.frames) - 1):
            frame_index = 0
        else:
            frame_index = frame_index + 1

    return np.array(np_frames)

模型架构

_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
conv1 (Conv3D)               (None, 16, 112, 112, 64)  5248      
_________________________________________________________________
pool1 (MaxPooling3D)         (None, 16, 56, 56, 64)    0         
_________________________________________________________________
conv2 (Conv3D)               (None, 16, 56, 56, 128)   221312    
_________________________________________________________________
pool2 (MaxPooling3D)         (None, 8, 28, 28, 128)    0         
_________________________________________________________________
conv3a (Conv3D)              (None, 8, 28, 28, 256)    884992    
_________________________________________________________________
conv3b (Conv3D)              (None, 8, 28, 28, 256)    1769728   
_________________________________________________________________
pool3 (MaxPooling3D)         (None, 4, 14, 14, 256)    0         
_________________________________________________________________
conv4a (Conv3D)              (None, 4, 14, 14, 512)    3539456   
_________________________________________________________________
conv4b (Conv3D)              (None, 4, 14, 14, 512)    7078400   
_________________________________________________________________
pool4 (MaxPooling3D)         (None, 2, 7, 7, 512)      0         
_________________________________________________________________
conv5a (Conv3D)              (None, 2, 7, 7, 512)      7078400   
_________________________________________________________________
conv5b (Conv3D)              (None, 2, 7, 7, 512)      7078400   
_________________________________________________________________
zeropad5 (ZeroPadding3D)     (None, 2, 8, 8, 512)      0         
_________________________________________________________________
pool5 (MaxPooling3D)         (None, 1, 4, 4, 512)      0         
_________________________________________________________________
flatten_1 (Flatten)          (None, 8192)              0         
_________________________________________________________________
fc6 (Dense)                  (None, 4096)              33558528  
_________________________________________________________________
dropout_1 (Dropout)          (None, 4096)              0         
_________________________________________________________________
fc7 (Dense)                  (None, 4096)              16781312  
_________________________________________________________________
dropout_2 (Dropout)          (None, 4096)              0         
_________________________________________________________________
nfc8 (Dense)                 (None, 2)                 8194      
=================================================================
Total params: 78,003,970
Trainable params: 78,003,970
Non-trainable params: 0
_________________________________________________________________
None

培训可视化

Answer 1

我认为错误在于损失函数和最后一个密集层。如模型摘要中所提供，最后一个Dense层是

nfc8 (Dense) (None, 2)

输出形状为（None，2）表示该图层具有2个单位。如前所述，您需要将GIF分为正值或负值。

对GIF进行分类可能是二进制分类问题，也可能是多类分类问题（具有两个类）。

二进制分类在最后一个密集层中只有1个单位具有S型激活功能。但是，这里的模型在最后一个Dense层中有2个单位。

因此，该模型是一个多类分类器，但是您给出了binary_crossentropy的损失函数，该函数用于二进制分类器（最后一层中有一个单元）。

因此，用categorical_crossentropy代替损失应该可以。或编辑最后一个Dense层并更改单位数和激活功能。

希望这会有所帮助。

为什么我的CNN过度拟合，我该如何解决？

1 个答案: