Question

我正在尝试重现https://www.tensorflow.org/tutorials/mnist/beginners/

的结果

所以我设计了几个函数来处理这两个训练步骤：

def layer_computation(previous_layer_output, weights, bias, activation):
    return activation(tf.add(tf.matmul(previous_layer_output, weights), bias))

def multilayer_perceptron_forward(x, weights, biaises, activations):
    return reduce(lambda output_layer, args: layer_computation(output_layer, *args),
              zip(weights, biaises, activations), x)

通过将这两个功能用于培训

def training(session,
         features, labels,
         mlp,
         # cost = (tf.reduce_mean, ),
         optimizer=tf.train.GradientDescentOptimizer,
         epochs=100, learning_rate=0.001, display=100):

x = tf.placeholder("float")
y = tf.placeholder("float")
weights, biases, activations = mlp
pred = multilayer_perceptron_forward(x, weights, biases, activations)
cost = tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(logits=pred, labels=y))
opti = optimizer(learning_rate).minimize(cost)

init = tf.global_variables_initializer()
session.run(init)
for i in range(1, epochs + 1):
    batch_size = 100
    avg_cost = 0
    number_of_bacth = int(features.shape[0]/batch_size)
    for j in range(number_of_bacth):
        my_x = features[j*100:(j+1)*100, :]
        my_y = labels[j*100:(j+1)*100, :]
        _, c = session.run([opti, cost], feed_dict={x: my_x,
                                                    y: my_y})
        avg_cost += c/number_of_bacth

    if i % display == 0:
        print("Epoch {i} cost = {cost}".format(i=i, cost=avg_cost))

优化以2.3的成本停止......总体精度为10％，而在示例中，它接近于零，精度接近96％。有没有人对这种奇特的行为有解释？

PS，当我在示例中的源代码中使用 layer_computation 时，我也会以2.3的成本陷入困境。

Answer 1

我发现错误，我试图在最后一层执行反向传播。在交叉验证时，这个问题可能会更好。

当激活步骤处于函数中时，tensorflow停止优化

1 个答案: