Question

tensorflow告诉我，当我使用动量优化器时，变量的动量部分是未启动的。当我使用gradientDescent优化器时，工作正常。

以下是堆栈跟踪的相关部分：

tensorflow.python.framework.errors.FailedPreconditionError: Attempting to use uninitialized value fc3/biases/Momentum
     [[Node: Momentum/update_fc3/biases/ApplyMomentum = ApplyMomentum[T=DT_FLOAT, _class=["loc:@fc3/biases"], use_locking=false, _device="/job:localhost/replica:0/task:0/cpu:0"](fc3/biases, fc3/biases/Momentum, Momentum/learning_rate, gradients/fc3/logits_grad/tuple/control_dependency_1, Momentum/momentum)]]
Caused by op u'Momentum/update_fc3/biases/ApplyMomentum', defined at:
...
    train_op = vgg.optimizer.minimize(vgg.loss, global_step=vgg.global_step)

我认为代码是正确的，它在初始化所有变量op之前定义了所有层的操作，等等。如果不是，GradientDescent优化器将无法正常工作？

跟进@etarion评论，这里是代码草图，以

开头

def train(args):
    datareader = # object to read data - no tensorflow code/import

    with tf.Graph().as_default():
        with_graph(datareader, args)

然后with_graph做

def with_graph(datareader, args):
    num_outputs = datareader.num_outputs()
    img_orig = tf.placeholder(tf.float32, shape=datareader.features_placeholder_shape())
    img_vgg16 = preprocess.imgbatch_2_vgg16(imgs=img_orig, channel_mean=8.46)
    labels_placeholder = tf.placeholder(tf.float32, shape=(None, num_outputs))
    vgg = vgg16(imgs=img_vgg16, weights=None, sess=None, trainable=args.trainable, stop_at_fc2=args.fc2)
    add_loss(vgg, labels_placeholder, num_outputs, args)
    add_optimizer(vgg, args)

    sess = tf.Session(config=tf.ConfigProto(intra_op_parallelism_threads = 12))    
    init = tf.initialize_all_variables()
    sess.run(init)


    validation_imgs_orig, validation_labels = datareader.get_validation_set()
    validation_imgs_vgg16 = sess.run(img_vgg16, {img_orig: validation_imgs_orig})
    validation_feed_dict = {img_vgg16:validation_imgs_vgg16,
                            labels_placeholder:validation_labels}    
    train_op = vgg.optimizer.minimize(vgg.loss, global_step=vgg.global_step)

    print("Starting training.")
    sys.stdout.flush()
    for step_number in range(3):
        t0 = time.time()
        train_imgs, train_labels = datareader.get_next_minibatch()
        train_feed_dict = {img_orig: train_imgs,
                           labels_placeholder:train_labels}
        sess.run(train_op, feed_dict=train_feed_dict)        
        print("step %3d took %.2f sec." % (step_number, time.time()-t0))
        sys.stdout.flush()

Answer 1

梯度下降优化器没有内部变量，具有动量。不知怎的，你没有初始化动量的状态（不能在没有代码的情况下告诉原因）。这样做的方法是在运行图形之前（在将优化器添加到图形之后）初始化所有变量，或者，如果要在初始化时明确，请使用{{3优化器的方法来获取构成优化器内部状态的变量。

Answer 2

问题是我在初始化所有变量后从优化器最小化函数定义了训练操作 - 一旦我在初始化它所有变量之前移动它。

张量流量动量部分变量未初始化？

2 个答案: