Question

我正在尝试使用以下代码打印出训练前后的网络权重：

weights = [v for v in tf.trainable_variables() if v.name == 'dense1/kernel:0'][0]
print(sess.run(weights))

但是，这些值完全不变。

当我尝试通过打印精度以及权重进行调试时，可以看到精度正在提高，但是权重保持不变。训练时的输出如下：

权重= [-0.07634658 -0.03764156] acc = 0.1551000028848648

权重= [-0.07634658 -0.03764156] acc = 0.4083999991416931

权重= [-0.07634658 -0.03764156] acc = 0.4812999963760376

权重= [-0.07634658 -0.03764156] acc = 0.3167000114917755

权重= [-0.07634658 -0.03764156] acc = 0.49880000948905945

权重= [-0.07634658 -0.03764156] acc = 0.42320001125335693

权重= [-0.07634658 -0.03764156] acc = 0.4494999945163727

权重= [-0.07634658 -0.03764156] acc = 0.578000009059906

权重= [-0.07634658 -0.03764156] acc = 0.6047999858856201

是错误吗？还是我正确打印了砝码？

下面是我尝试调试的简单模型

import tensorflow as tf
from tensorflow.examples.tutorials.mnist import input_data
import os

os.environ['CUDA_VISIBLE_DEVICES']='3'

config = tf.ConfigProto()
config.gpu_options.per_process_gpu_memory_fraction = 0.1

mnist = input_data.read_data_sets("MNIST_data/", one_hot=True)

X = tf.placeholder(tf.float32, [None, 784])
y_true = tf.placeholder(tf.int32, [None, 10]) 

layer1 = tf.layers.dense(X, 2, name='dense1')
output = tf.layers.dense(layer1, 10, name='dense2')

cross_entropy = tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits_v2(y_true, output))

optimizer = tf.train.GradientDescentOptimizer(learning_rate=0.5)
train = optimizer.minimize(cross_entropy)

init = tf.global_variables_initializer()

sess = tf.Session(config=config)
sess.run(init)

weights = [v for v in tf.trainable_variables() if v.name == 'dense1/kernel:0'][0]

print("weights before training",sess.run(weights))


for step in range(1000):
    batch_x, batch_y = mnist.train.next_batch(100)
    sess.run(train, feed_dict={X:batch_x, y_true:batch_y})

    if step % 50 ==0:
        weights = [v for v in tf.trainable_variables() if v.name == 'dense1/kernel:0'][0]
        print("weights = ", sess.run(weights[0]))
        correct_prediction = tf.equal(tf.argmax(output,1), tf.argmax(y_true,1))
        acc = tf.reduce_mean(tf.cast(correct_prediction, tf.float32))
        print("acc = ",sess.run(acc, feed_dict={X:mnist.test.images, y_true:mnist.test.labels}))




weights_after = [v for v in tf.trainable_variables() if v.name == 'dense1/kernel:0'][0]

print("weights after training",sess.run(weights_after))

Answer 1

“ dense1 / kernel”（W1）中的权重没有变化，但是在“ dense2 / kernel”（W2）中，权重发生了变化。 W2更新是精度更新的原因。这意味着W1并未通过梯度下降进行训练，但W2却进行了训练。买方法，如果不使用sess.close()

，请最后添加with tf.Session() as sess:

Answer 2

这看起来不错，但是它错过了Model的构建。型号（输入= X，输出=输出）

训练前后Tensorflow权重不会改变

2 个答案: