Question

我有以下问题：

我在Keras有一个自动编码器，并训练了几个时代。培训概述显示验证MAE为0.0422，MSE为0.0024。但是，如果我然后调用network.predict并手动计算验证错误，我得到0.035和0.0024。

人们会认为我对MAE的手动计算是不正确的，但奇怪的是，如果我使用身份模型（只输出您输入的内容）并使用它来评估预测值，则相同的误差值是返回我的手动计算。代码如下：

input = Input(shape=(X_train.shape[1], ))
encoded = Dense(50, activation='relu', activity_regularizer=regularizers.l1(10e-5))(input)
encoded = Dense(50, activation='relu', activity_regularizer=regularizers.l1(10e-5))(encoded)
encoded = Dense(50, activation='relu', activity_regularizer=regularizers.l1(10e-5))(encoded)
decoded = Dense(50, activation='relu', activity_regularizer=regularizers.l1(10e-5))(encoded)
decoded = Dense(50, activation='relu', activity_regularizer=regularizers.l1(10e-5))(decoded)
decoded = Dense(X_train.shape[1], activation='sigmoid')(decoded)
network = Model(input, decoded)

# sgd = SGD(lr=8, decay=1e-6)
# network.compile(loss='mean_squared_error', optimizer='adam')
network.compile(loss='mean_absolute_error', optimizer='adam', metrics=['mse'])

# Fitting the data
network.fit(X_train, X_train, epochs=2, batch_size=1, shuffle=True, validation_data=(X_valid, X_valid),
            callbacks=[EarlyStopping(monitor='val_loss', min_delta=0.00001, patience=20, verbose=0, mode='auto')])


# Results
recon_valid = network.predict(X_valid, batch_size=1)
score2 = network.evaluate(X_valid, X_valid, batch_size=1, verbose=0)
print('Network evaluate result: mae={}, mse={}'.format(*score2))

x = Input((X_train.shape[1],))
m = Model(x, x)
m.compile(loss='mean_absolute_error', optimizer='adam', metrics=['mse'])
score1 = m.evaluate(recon_valid, X_valid, batch_size=1, verbose=0)
print('Identity evaluate result: mae={}, mse={}'.format(*score1))

errors_test = np.absolute(X_valid - recon_valid)
print("Manual MAE: {}".format(np.average(errors_test)))
errors_test = np.square(X_valid - recon_valid)
print("Manual MSE: {}".format(np.average(errors_test)))

其中输出以下内容：

Train on 282 samples, validate on 94 samples
Epoch 1/2
2018-04-18 17:24:01.464947: I C:\tf_jenkins\workspace\rel-win\M\windows\PY\36\tensorflow\core\platform\cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX AVX2
282/282 [==============================] - 0s - loss: 0.0861 - mean_squared_error: 0.0187 - val_loss: 0.0451 - val_mean_squared_error: 0.0025
Epoch 2/2
282/282 [==============================] - 0s - loss: 0.0440 - mean_squared_error: 0.0025 - val_loss: 0.0422 - val_mean_squared_error: 0.0024
Network evaluate result: mae=0.04216482736011769, mse=0.0024067993242382767
Identity evaluate result: mae=0.03506102238563781, mse=0.0024067993242382767
Manual MAE: 0.03506102412939072
Manual MSE: 0.002406799467280507

我知道我的手动计算是正确的，因为身份模型（m）返回相同的值。如果network.evaluate（X_valid，X_valid）以某种方式使用与network.predict（X_valid）返回的值不同的值，那么MAE值差异的唯一可能解释是，但是MSE也将是不同的。

这让我完全感到困惑，认为Keras MAE计算中可能存在错误。有没有人之前有这个问题或有任何想法如何解决？我正在使用Tensorflow后端。任何帮助将不胜感激！

编辑：我几乎可以肯定这是一个错误。如果我保持loss ='mae'但也添加metrics = ['mse'，'mae']，则指标返回的MAE与我的手动计算和身份模型相同。对于MSE也是如此：如果我设置了loss ='mse'，则度量返回的MSE与损失不同。

Answer 1

事实证明，由于正规化，损失应该与度量不同。使用正则化，损失更高（在我的情况下），因为当节点不像指定的那样活跃时，正则化会增加损失。指标不考虑这一点，因此会返回一个不同的值，该值等于手动计算错误时的值。

Answer 2

The metrics during training and validation are different because of different reasons:

The dataset is different
During trainning the weights are changing in every step so the metrics are changing too
The metric during training is of the current batch of data or a running average of the metrics of the last batches. For the evaluation, the metric is for the whole dataset.

Keras预测给出的误差与评估不同，损失与指标不同

2 个答案: