Question

我正在尝试了解在时代末尾在keras进度栏中显示的精度“ acc ”：

13/13 [==============================]-0s 76us / step-损耗：0.7100- acc ：0.4615

在一个时代结束时，应该是所有训练样本的模型预测的准确性。但是，当在相同的训练样本上评估模型时，实际准确性可能会非常不同。

下面是MLP for binary classification from keras webpage的改编示例。一个简单的顺序神经网络对随机生成的数字进行二进制分类。批处理大小与训练示例的数量相同（13），因此每个时期仅包含一个步骤。由于损耗设置为binary_crossentropy，因此，为了进行精度计算，使用了metrics.py中定义的binary_accuracy。 MyEval类定义了回调，该回调在每个时期的末尾被调用。它使用两种方法来计算训练数据的准确性：a）模型评估和b）模型预测以获取预测，然后使用与keras binary_accuracy函数中几乎相同的代码。这两个精度是一致的，但是大多数时间与进度栏中的精度不同。为什么它们不同？是否可以计算与进度栏中相同的精度？还是我的假设有误？

import numpy as np
from keras.models import Sequential
from keras.layers import Dense, Dropout
from keras import callbacks

np.random.seed(1) # fix random seed for reproducibility 
# Generate dummy data
x_train = np.random.random((13, 20))
y_train = np.random.randint(2, size=(13, 1))

model = Sequential()
model.add(Dense(64, input_dim=20, activation='relu'))
model.add(Dropout(0.5))
model.add(Dense(64, activation='relu'))
model.add(Dropout(0.5))
model.add(Dense(1, activation='sigmoid'))

model.compile(loss='binary_crossentropy',
              optimizer='rmsprop',
              metrics=['accuracy'])

class MyEval(callbacks.Callback):
    def on_epoch_end(self, epoch, logs=None):
        my_accuracy_1 = self.model.evaluate(x_train, y_train, verbose=0)[1]
        y_pred = self.model.predict(x_train)
        my_accuracy_2 = np.mean(np.equal(y_train, np.round(y_pred)))
        print("my accuracy 1: {}".format(my_accuracy_1))
        print("my accuracy 2: {}".format(my_accuracy_2))

my_eval = MyEval()

model.fit(x_train, y_train,
          epochs=5,
          batch_size=13,
          callbacks=[my_eval],
          shuffle=False)

以上代码的输出：

13/13 [==============================] - 0s 25ms/step - loss: 0.7303 - acc: 0.5385
my accuracy 1: 0.5384615659713745
my accuracy 2: 0.5384615384615384
Epoch 2/5
13/13 [==============================] - 0s 95us/step - loss: 0.7412 - acc: 0.4615
my accuracy 1: 0.9230769276618958
my accuracy 2: 0.9230769230769231
Epoch 3/5
13/13 [==============================] - 0s 77us/step - loss: 0.7324 - acc: 0.3846
my accuracy 1: 0.9230769276618958
my accuracy 2: 0.9230769230769231
Epoch 4/5
13/13 [==============================] - 0s 72us/step - loss: 0.6543 - acc: 0.5385
my accuracy 1: 0.9230769276618958
my accuracy 2: 0.9230769230769231
Epoch 5/5
13/13 [==============================] - 0s 76us/step - loss: 0.6459 - acc: 0.6923
my accuracy 1: 0.8461538553237915
my accuracy 2: 0.8461538461538461

使用：Python 3.5.2，tensorflow-gpu == 1.14.0 Keras == 2.2.4 numpy == 1.15.2

Answer 1

我认为这与Dropout的使用有关。仅在训练期间启用辍学，而在评估或预测期间则不启用。因此，在训练和评估/预测过程中的准确性差异。

此外，显示在栏中的训练准确度显示了训练时期内的平均准确度，是每个批次之后计算出的批次准确度的平均值。请记住，每次批次后都要对模型参数进行调整，以使末尾栏中显示的准确度与纪元完成后的准确度不完全匹配（因为训练准确度是根据每个批次，并且对所有批次使用相同的参数计算验证准确性）。

这是您的示例，具有更多数据（因此有一个以上的纪元），并且没有丢失：

import numpy as np
from keras.models import Sequential
from keras.layers import Dense, Dropout
from keras import callbacks

np.random.seed(1) # fix random seed for reproducibility 
# Generate dummy data
x_train = np.random.random((200, 20))
y_train = np.random.randint(2, size=(200, 1))

model = Sequential()
model.add(Dense(64, input_dim=20, activation='relu'))
# model.add(Dropout(0.5))
model.add(Dense(64, activation='relu'))
# model.add(Dropout(0.5))
model.add(Dense(1, activation='sigmoid'))

model.compile(loss='binary_crossentropy',
              optimizer='rmsprop',
              metrics=['accuracy'])

class MyEval(callbacks.Callback):
    def on_epoch_end(self, epoch, logs=None):
        my_accuracy_1 = self.model.evaluate(x_train, y_train, verbose=0)[1]
        y_pred = self.model.predict(x_train)
        my_accuracy_2 = np.mean(np.equal(y_train, np.round(y_pred)))
        print("my accuracy 1 after epoch {}: {}".format(epoch + 1,my_accuracy_1))
        print("my accuracy 2 after epoch {}: {}".format(epoch + 1,my_accuracy_2))


my_eval = MyEval()

model.fit(x_train, y_train,
          epochs=5,
          batch_size=13,
          callbacks=[my_eval],
          shuffle=False)

输出为：

Train on 200 samples
Epoch 1/5
my accuracy 1 after epoch 1: 0.5450000166893005
my accuracy 2 after epoch 1: 0.545
200/200 [==============================] - 0s 2ms/sample - loss: 0.6978 - accuracy: 0.5350
Epoch 2/5
my accuracy 1 after epoch 2: 0.5600000023841858
my accuracy 2 after epoch 2: 0.56
200/200 [==============================] - 0s 383us/sample - loss: 0.6892 - accuracy: 0.5550
Epoch 3/5
my accuracy 1 after epoch 3: 0.5799999833106995
my accuracy 2 after epoch 3: 0.58
200/200 [==============================] - 0s 496us/sample - loss: 0.6844 - accuracy: 0.5800
Epoch 4/5
my accuracy 1 after epoch 4: 0.6000000238418579
my accuracy 2 after epoch 4: 0.6
200/200 [==============================] - 0s 364us/sample - loss: 0.6801 - accuracy: 0.6150
Epoch 5/5
my accuracy 1 after epoch 5: 0.6050000190734863
my accuracy 2 after epoch 5: 0.605
200/200 [==============================] - 0s 393us/sample - loss: 0.6756 - accuracy: 0.6200

新纪元之后的验证准确性非常类似于现在新纪元结束时的平均训练准确性。

Answer 2

您正在使用随机图像和随机标签。我创建了一个包含真实图像和随机标签的模型。我得到类似的结果。因为随机图像和标签不是线性相关的。我建议您使用包含真实图像和真实标签的数据集。

进度栏中显示的角膜准确性如何计算？从哪些输入中计算得出？如何复制它？

2 个答案: