Question

我用张量流写了一个简单的神经网络。在培训期间，我发现小批量损失保持不变，但小批量精度不同。

示例：

Step 62000, Mini-batch Loss = 0.6931, Training Accuracy = 0.910
Step 63000, Mini-batch Loss = 0.6931, Training Accuracy = 0.909
Step 64000, Mini-batch Loss = 0.6931, Training Accuracy = 0.918
Step 65000, Mini-batch Loss = 0.6931, Training Accuracy = 0.910

对我来说似乎有点奇怪。好不好？你怎么看？只是一个小评论：我正在使用亚当和批处理...所以不是所有的训练样例都是在一次迭代中计算出来的。

以下是代码：

layers_output = {'A0': X}
for i in range(1, hidden_layers + 2):
    Z = tf.add(tf.matmul(layers_output['A' + str(i - 1)], weights['W' + str(i)]), biases['b' + str(i)])
    if i == hidden_layers + 1:
        layers_output['A' + str(i)] = tf.sigmoid(Z)
    else:
        layers_output['A' + str(i)] = tf.tanh(Z)
logits = layers_output['A' + str(hidden_layers + 1)]
prediction = tf.round(logits)

# Define loss and optimizer
loss_op = tf.reduce_mean(tf.nn.sigmoid_cross_entropy_with_logits(logits=logits, labels=Y))
train_op = tf.train.AdamOptimizer(learning_rate=learning_rate, beta1=beta1, beta2=beta2).minimize(loss_op)

# Evaluate model
correct_pred = tf.equal(prediction, Y)
accuracy = tf.reduce_mean(tf.cast(correct_pred, tf.float32))

init = tf.global_variables_initializer()

with tf.Session() as sess:
    sess.run(init)

    mini_batches = divide_into_mini_batches(X_train, Y_train, mini_batch_size)
    for step in range(1, num_steps + 1):
        batch_x = mini_batches[step % len(mini_batches)][0]
        batch_y = mini_batches[step % len(mini_batches)][1]
        # Run optimization op (backprop)
        sess.run(train_op, feed_dict={X: batch_x, Y: batch_y})
        if step % display_step == 0 or step == 1:
            # Calculate batch loss and accuracy
            loss, acc = sess.run([loss_op, accuracy], feed_dict={X: batch_x, Y: batch_y})
            print("Step " + str(step) + ", Mini-batch Loss = " +
                  "{:.4f}".format(loss) + ", Training Accuracy = " +
                  "{:.3f}".format(acc))

由于

神经网络训练：小批量丢失相同，小批量精度不同

0 个答案: