Question

我有一个TensorFlow（TF）模型，我想恢复并重新训练一些参数。我知道tf.get_operation_by_name('name of the optimizer')检索用于在模型存储之前训练模型的原始优化器。但是，我不知道如何传递我希望优化器重新训练的新TF变量列表！

此示例有助于说明我想要做的事情：

learning_rate = 0.0001
training_iters = 60000
batch_size = 64
display_step = 20
ImVecDim = 784# The number of elements in a an image vector (flattening a  28x28 2D image)
NumOfClasses = 10
dropout = 0.8

with tf.Session() as sess:
   LoadMod = tf.train.import_meta_graph('simple_mnist.ckpt.meta')  # This object loads the model
   LoadMod.restore(sess, tf.train.latest_checkpoint('./')) # Loading weights and biases and other stuff to the model
   g = tf.get_default_graph()

   # Variables to be retrained:
   wc2 = g.get_tensor_by_name('wc2:0')
   bc2 = g.get_tensor_by_name('bc2:0')
   wc3 = g.get_tensor_by_name('wc3:0')
   bc3 = g.get_tensor_by_name('bc3:0')
   wd1 = g.get_tensor_by_name('wd1:0')
   bd1 = g.get_tensor_by_name('bd1:0')
   wd2 = g.get_tensor_by_name('wd2:0')
   bd2 = g.get_tensor_by_name('bd2:0')
   out_w = g.get_tensor_by_name('out_w:0')
   out_b = g.get_tensor_by_name('out_b:0')
   VarToTrain = [wc2,wc3,wd1,wd2,out_w,bc2,bc3,bd1,bd2,out_b]

   # Retrieving the optimizer:
   Opt = tf.get_operation_by_name('Adam')

  # Retraining:
  X = g.get_tensor_by_name('ImageIn:0')
  Y = g.get_tensor_by_name('LabelIn:0')
  KP = g.get_tensor_by_name('KeepProb:0')
  accuracy = g.get_tensor_by_name('NetAccuracy:0')
  cost = g.get_tensor_by_name('loss:0')
  step = 1
  while step * batch_size < training_iters:
            batch_xs, batch_ys = mnist.train.next_batch(batch_size)
  #########################################################################
  #     Here I want to pass (VarToTrain) to the optimizer (Opt)!          #
  #########################################################################          
            if step % display_step == 0:
                acc = sess.run(accuracy, feed_dict={X: batch_xs, Y: batch_ys, KP: 1.})
                loss = sess.run(cost, feed_dict={X: batch_xs, Y: batch_ys, KP: 1.})
                print("Iter " + str(step * batch_size) + ", Minibatch Loss= " + "{:.6f}".format(
                    loss) + ", Training Accuracy= " + "{:.5f}".format(acc))
            step += 1
  feed_dict = {X: mnist.test.images[:256], Y: mnist.test.labels[:256], KP: 1.0}
  ModelAccuracy = sess.run(accuracy, feed_dict)
  print('Retraining finished'+', Test Accuracy = %f' %ModelAccuracy)

Answer 1

好吧，我还没有找到一种方法来完成我想要的事情，但我找到了解决问题的方法;我没有将新的变量列表传递给原始优化器，而是使用传递给其minimize()方法的变量定义了一个新的优化器。代码如下：

learning_rate = 0.0001
training_iters = 60000
batch_size = 64
display_step = 20
ImVecDim = 784# The number of elements in a an image vector (flattening a  28x28 2D image)
NumOfClasses = 10
dropout = 0.8

with tf.Session() as sess:
   LoadMod = tf.train.import_meta_graph('simple_mnist.ckpt.meta')  # This object loads the model
   LoadMod.restore(sess, tf.train.latest_checkpoint('./')) # Loading weights and biases and other stuff to the model
   g = tf.get_default_graph()
# Retraining:
  X = g.get_tensor_by_name('ImageIn:0')
  Y = g.get_tensor_by_name('LabelIn:0')
  KP = g.get_tensor_by_name('KeepProb:0')
  accuracy = g.get_tensor_by_name('NetAccuracy:0')
  cost = g.get_tensor_by_name('loss:0')

######################## Producing a list and defining a new optimizer ####################################
  VarToTrain = g.get_collection_ref('trainable__variables')
  del VarToTrain[0] # Removing a variable from the list
  del VarToTrain[5] # Removing another variable from the list
  optimizer = tf.train.GradientDescentOptimizer(learning_rate=learning_rate).\
                minimize(cost,var_list= VarToTrain)
##########################################################################################
  step = 1
  while step * batch_size < training_iters:
      batch_xs, batch_ys = mnist.train.next_batch(batch_size)
      sess.run(optimizer, feed_dict={X: batch_xs, Y: batch_ys, KP: dropout})
      if step % display_step == 0:
            acc = sess.run(accuracy, feed_dict={X: batch_xs, Y: batch_ys, KP: 1.})
            loss = sess.run(cost, feed_dict={X: batch_xs, Y: batch_ys, KP: 1.})
            print("Iter " + str(step * batch_size) + ", Minibatch Loss= " + "{:.6f}".format(
                        loss) + ", Training Accuracy= " + "{:.5f}".format(acc))
            step += 1
   feed_dict = {X: mnist.test.images[:256], Y: mnist.test.labels[:256], KP: 1.0}
   ModelAccuracy = sess.run(accuracy, feed_dict)
   print('Retraining finished'+', Test Accuracy = %f' %ModelAccuracy)

上面的代码完成了这项工作，但它有一些问题！首先，出于某种原因，每次我将原始的{$ 1}}定义为类似的优化器时，我都会收到错误消息。我可以定义的唯一优化器没有TF抛出错误消息是tf.train.AdamOtimizer()。该解决方案的另一个问题是它的不便之处;为了生成我想要训练的变量列表，我首先要使用tf.train.GradientDescentOptimizer()生成所有可训练变量的列表，打印出来，记住列表中不需要的变量的位置，然后删除他们一个接一个地用VarToTrain = g.get_collection_ref('trainable_variables')方法!!必须有一种更优雅的方式来做到这一点。我所做的只适用于小型网络!!

在恢复的TensorFlow模型中训练一些变量

1 个答案: