培训和验证损失会达到一定的饱和值,并且不会进一步降低

时间:2019-02-05 18:10:10

标签: tensorflow keras deep-learning conv-neural-network

我正在尝试建立一个网络来检测脸部的68个界标(x,y)。训练和验证图像的320x320x3规格为-0.5至0.5。我的标签是136个logit,每个在0到1.0之间,对应于X->(0,320); Y->(0,320)。损失函数是keras“ root_mean_square”。训练数据集中的图像数量约为5k。在训练过程中,我的训练和验证损失从大约6.0开始,并在100次迭代中减少到大约0.0022,但随后似乎在该水平达到饱和,并且没有任何降低。我已经尝试了2000次迭代。从输出看,似乎网络学会了在帧中心以脸部形状输出68个点,而与实际位置无关。

我正在使用生成器来获取数据,并使用sklearn.utils.shuffle()来确保我的数据被正确地重新整理。

一些帖子建议网络可能过拟合,因为它对于一个简单的问题是如此复杂,因此我尝试了一个约10层的非常简单的网络和约20层的复杂网络,但我的结果仍然相同。我的当前网络如下所示,我使用了2个跳过连接,3个辍学和l2正则化器来确保我的网络不会过拟合。拟合不足应该不是问题,因为我已经尝试对网络进行多达2000次迭代的训练。

对于如何解决此问题的任何建议,我们将不胜感激。谢谢!

__________________________________________________________________________________________________
Layer (type)                    Output Shape         Param #     Connected to                     
==================================================================================================
input_1 (InputLayer)            (None, 320, 320, 3)  0                                            
__________________________________________________________________________________________________
conv2d_1 (Conv2D)               (None, 320, 320, 3)  228         input_1[0][0]                    
__________________________________________________________________________________________________
batch_normalization_1 (BatchNor (None, 320, 320, 3)  12          conv2d_1[0][0]                   
__________________________________________________________________________________________________
activation_1 (Activation)       (None, 320, 320, 3)  0           batch_normalization_1[0][0]      
__________________________________________________________________________________________________
max_pooling2d_1 (MaxPooling2D)  (None, 160, 160, 3)  0           activation_1[0][0]               
__________________________________________________________________________________________________
conv2d_2 (Conv2D)               (None, 160, 160, 8)  608         max_pooling2d_1[0][0]            
__________________________________________________________________________________________________
batch_normalization_2 (BatchNor (None, 160, 160, 8)  32          conv2d_2[0][0]                   
__________________________________________________________________________________________________
activation_2 (Activation)       (None, 160, 160, 8)  0           batch_normalization_2[0][0]      
__________________________________________________________________________________________________
max_pooling2d_2 (MaxPooling2D)  (None, 80, 80, 8)    0           activation_2[0][0]               
__________________________________________________________________________________________________
conv2d_3 (Conv2D)               (None, 80, 80, 16)   1168        max_pooling2d_2[0][0]            
__________________________________________________________________________________________________
conv2d_4 (Conv2D)               (None, 80, 80, 16)   2320        conv2d_3[0][0]                   
__________________________________________________________________________________________________
batch_normalization_3 (BatchNor (None, 80, 80, 16)   64          conv2d_4[0][0]                   
__________________________________________________________________________________________________
activation_3 (Activation)       (None, 80, 80, 16)   0           batch_normalization_3[0][0]      
__________________________________________________________________________________________________
max_pooling2d_4 (MaxPooling2D)  (None, 40, 40, 16)   0           activation_3[0][0]               
__________________________________________________________________________________________________
conv2d_5 (Conv2D)               (None, 40, 40, 32)   4640        max_pooling2d_4[0][0]            
__________________________________________________________________________________________________
conv2d_6 (Conv2D)               (None, 40, 40, 32)   9248        conv2d_5[0][0]                   
__________________________________________________________________________________________________
conv2d_7 (Conv2D)               (None, 40, 40, 32)   9248        conv2d_6[0][0]                   
__________________________________________________________________________________________________
batch_normalization_4 (BatchNor (None, 40, 40, 32)   128         conv2d_7[0][0]                   
__________________________________________________________________________________________________
activation_4 (Activation)       (None, 40, 40, 32)   0           batch_normalization_4[0][0]      
__________________________________________________________________________________________________
max_pooling2d_6 (MaxPooling2D)  (None, 20, 20, 32)   0           activation_4[0][0]               
__________________________________________________________________________________________________
dropout_1 (Dropout)             (None, 20, 20, 32)   0           max_pooling2d_6[0][0]            
__________________________________________________________________________________________________
conv2d_8 (Conv2D)               (None, 20, 20, 64)   18496       dropout_1[0][0]                  
__________________________________________________________________________________________________
conv2d_9 (Conv2D)               (None, 20, 20, 64)   36928       conv2d_8[0][0]                   
__________________________________________________________________________________________________
max_pooling2d_3 (MaxPooling2D)  (None, 20, 20, 16)   0           conv2d_4[0][0]                   
__________________________________________________________________________________________________
conv2d_10 (Conv2D)              (None, 20, 20, 64)   36928       conv2d_9[0][0]                   
__________________________________________________________________________________________________
concatenate_1 (Concatenate)     (None, 20, 20, 80)   0           max_pooling2d_3[0][0]            
                                                                 conv2d_10[0][0]                  
__________________________________________________________________________________________________
batch_normalization_5 (BatchNor (None, 20, 20, 80)   320         concatenate_1[0][0]              
__________________________________________________________________________________________________
activation_5 (Activation)       (None, 20, 20, 80)   0           batch_normalization_5[0][0]      
__________________________________________________________________________________________________
max_pooling2d_7 (MaxPooling2D)  (None, 10, 10, 80)   0           activation_5[0][0]               
__________________________________________________________________________________________________
dropout_2 (Dropout)             (None, 10, 10, 80)   0           max_pooling2d_7[0][0]            
__________________________________________________________________________________________________
conv2d_11 (Conv2D)              (None, 10, 10, 128)  92288       dropout_2[0][0]                  
__________________________________________________________________________________________________
conv2d_12 (Conv2D)              (None, 10, 10, 128)  147584      conv2d_11[0][0]                  
__________________________________________________________________________________________________
max_pooling2d_5 (MaxPooling2D)  (None, 10, 10, 32)   0           conv2d_7[0][0]                   
__________________________________________________________________________________________________
conv2d_13 (Conv2D)              (None, 10, 10, 128)  147584      conv2d_12[0][0]                  
__________________________________________________________________________________________________
concatenate_2 (Concatenate)     (None, 10, 10, 160)  0           max_pooling2d_5[0][0]            
                                                                 conv2d_13[0][0]                  
__________________________________________________________________________________________________
batch_normalization_6 (BatchNor (None, 10, 10, 160)  640         concatenate_2[0][0]              
__________________________________________________________________________________________________
activation_6 (Activation)       (None, 10, 10, 160)  0           batch_normalization_6[0][0]      
__________________________________________________________________________________________________
max_pooling2d_8 (MaxPooling2D)  (None, 5, 5, 160)    0           activation_6[0][0]               
__________________________________________________________________________________________________
dropout_3 (Dropout)             (None, 5, 5, 160)    0           max_pooling2d_8[0][0]            
__________________________________________________________________________________________________
flatten_2 (Flatten)             (None, 4000)         0           dropout_3[0][0]                  
__________________________________________________________________________________________________
dense_3 (Dense)                 (None, 1024)         4097024     flatten_2[0][0]                  
__________________________________________________________________________________________________
dense_4 (Dense)                 (None, 136)          139400      dense_3[0][0]                    
==================================================================================================
Total params: 4,744,888
Trainable params: 4,744,290
Non-trainable params: 598
__________________________________________________________________________________________________

0 个答案:

没有答案