Question

我想将几个不同的输入传递到可重用的张量流体系结构（解码器）中。为此，我使用了一个for循环，在其中将我的输入输入模型。但是，我无法重用图层变量，而是为每次循环迭代创建变量。假设此代码：

import tensorflow as tf

for i in range(5):
    decoder(input=input, is_training=is_training)

解码器处于以下状态：

def decoder(self, input, is_training):

    with tf.variable_scope("physics", reuse=tf.AUTO_REUSE):
         latent = tf.expand_dims(latent, axis=1)
         latent = tf.expand_dims(latent, axis=1)

         x = latent

         """ Layer 1 """
         x = tf.layers.conv2d_transpose(x, filters=256, kernel_size=2, strides=1, activation='relu', padding='valid', name="transpose1_1", reuse=tf.AUTO_REUSE)
         x = tf.layers.batch_normalization(x, training=is_training, name="transpose_bn_1_1")

         """ Layer 2 """
         x = tf.layers.conv2d_transpose(x, filters=256, kernel_size=2, strides=2, activation='relu', padding='valid', name="transpose1_2", reuse=tf.AUTO_REUSE)
         x = tf.layers.batch_normalization(x, training=is_training, name="transpose_bn_1_2")

         ...

如果我现在使用

循环后立即输出变量

from pprint import pprint
pprint([n.name for n in tf.get_default_graph().as_graph_def().node])

我得到以下输出，表明我不在循环迭代中共享变量：

 'physics/transpose1_1/kernel/Initializer/random_uniform/shape',
 'physics/transpose1_1/kernel/Initializer/random_uniform/min',
 'physics/transpose1_1/kernel/Initializer/random_uniform/max',
 'physics/transpose1_1/kernel/Initializer/random_uniform/RandomUniform',
 'physics/transpose1_1/kernel/Initializer/random_uniform/sub',
 'physics/transpose1_1/kernel/Initializer/random_uniform/mul',
 'physics/transpose1_1/kernel/Initializer/random_uniform',
 'physics/transpose1_1/kernel',
 'physics/transpose1_1/kernel/Assign',
 'physics/transpose1_1/kernel/read',
 'physics/transpose1_1/bias/Initializer/zeros',
 'physics/transpose1_1/bias',
 'physics/transpose1_1/bias/Assign',
 'physics/transpose1_1/bias/read',
 'physics/transpose1_1/Shape',
 'physics/transpose1_1/strided_slice/stack',
 'physics/transpose1_1/strided_slice/stack_1',
 'physics/transpose1_1/strided_slice/stack_2',
 'physics/transpose1_1/strided_slice',
 'physics/transpose1_1/strided_slice_1/stack',
 'physics/transpose1_1/strided_slice_1/stack_1',
 'physics/transpose1_1/strided_slice_1/stack_2',
 'physics/transpose1_1/strided_slice_1',
 'physics/transpose1_1/strided_slice_2/stack',
 'physics/transpose1_1/strided_slice_2/stack_1',
 'physics/transpose1_1/strided_slice_2/stack_2',
 'physics/transpose1_1/strided_slice_2',
 'physics/transpose1_1/mul/y',
 'physics/transpose1_1/mul',
 'physics/transpose1_1/add/y',
 'physics/transpose1_1/add',
 'physics/transpose1_1/mul_1/y',
 'physics/transpose1_1/mul_1',
 'physics/transpose1_1/add_1/y',
 'physics/transpose1_1/add_1',
 'physics/transpose1_1/stack/3',
 'physics/transpose1_1/stack',
 'physics/transpose1_1/conv2d_transpose',
 'physics/transpose1_1/BiasAdd',
 'physics/transpose1_1/Relu',
 ...
 'physics_4/transpose1_1/Shape',
 'physics_4/transpose1_1/strided_slice/stack',
 'physics_4/transpose1_1/strided_slice/stack_1',
 'physics_4/transpose1_1/strided_slice/stack_2',
 'physics_4/transpose1_1/strided_slice',
 'physics_4/transpose1_1/strided_slice_1/stack',
 'physics_4/transpose1_1/strided_slice_1/stack_1',
 'physics_4/transpose1_1/strided_slice_1/stack_2',
 'physics_4/transpose1_1/strided_slice_1',
 'physics_4/transpose1_1/strided_slice_2/stack',
 'physics_4/transpose1_1/strided_slice_2/stack_1',
 'physics_4/transpose1_1/strided_slice_2/stack_2',
 'physics_4/transpose1_1/strided_slice_2',
 'physics_4/transpose1_1/mul/y',
 'physics_4/transpose1_1/mul',
 'physics_4/transpose1_1/add/y',
 'physics_4/transpose1_1/add',
 'physics_4/transpose1_1/mul_1/y',
 'physics_4/transpose1_1/mul_1',
 'physics_4/transpose1_1/add_1/y',
 'physics_4/transpose1_1/add_1',
 'physics_4/transpose1_1/stack/3',
 'physics_4/transpose1_1/stack',
 'physics_4/transpose1_1/conv2d_transpose',
 'physics_4/transpose1_1/BiasAdd',
 'physics_4/transpose1_1/Relu',

这是怎么回事？ tf.AUTO_REUSE标志不应该允许我在decoder时先初始化i==0并为所有迭代i>0重用我的变量吗？对于我在解码器中的每一层，都会重复执行以上操作。

我正在使用TensorFlow版本1.12.0。

谢谢。

Answer 1

您已经在for循环中重用了变量。图的节点不等于Variable。以下示例有多个节点，但只有一个Variable。

import tensorflow as tf

a = tf.Variable([2.0],name='a')
b = a+1
print([n.name for n in tf.get_default_graph().as_graph_def().node])

['a/initial_value', 'a', 'a/Assign', 'a/read', 'add/y', 'add']

您应该使用其他方式在代码中查看变量。

1。在理解的末尾添加if "Variable" in n.op

print([n.name for n in tf.get_default_graph().as_graph_def().node if "Variable" in n.op])

['a']

2。使用tf.global_variables()。

print(tf.global_variables())

[<tf.Variable 'a:0' shape=(1,) dtype=float32_ref>]

因此，您应该在代码中执行以下操作：

import tensorflow as tf

def decoder(latent, is_training):
    with tf.variable_scope("physics", reuse=tf.AUTO_REUSE):
        x = latent
        """ Layer 1 """
        x = tf.layers.conv2d_transpose(x, filters=256, kernel_size=2, strides=1, activation='relu', padding='valid', name="transpose1_1", reuse=tf.AUTO_REUSE)
        x = tf.layers.batch_normalization(x, training=is_training, name="transpose_bn_1_1")
        """ Layer 2 """
        x = tf.layers.conv2d_transpose(x, filters=256, kernel_size=2, strides=2, activation='relu', padding='valid', name="transpose1_2", reuse=tf.AUTO_REUSE)
        x = tf.layers.batch_normalization(x, training=is_training, name="transpose_bn_1_2")

for i in range(5):
    decoder(latent=tf.ones(shape=[64,7,7,256]) , is_training=True)

print([n.name  for n in tf.get_default_graph().as_graph_def().node if "Variable" in n.op])
# print(tf.global_variables())

['physics/transpose1_1/kernel', 'physics/transpose1_1/bias', 'physics/transpose_bn_1_1/gamma', 'physics/transpose_bn_1_1/beta', 'physics/transpose_bn_1_1/moving_mean', 'physics/transpose_bn_1_1/moving_variance', 'physics/transpose1_2/kernel', 'physics/transpose1_2/bias', 'physics/transpose_bn_1_2/gamma', 'physics/transpose_bn_1_2/beta', 'physics/transpose_bn_1_2/moving_mean', 'physics/transpose_bn_1_2/moving_variance']

Answer 2

TF从图层名称中构建一个变量名称。然后，当它尝试创建变量时，它将检查变量是否已经存在。如果这样，它将抛出异常，除非您指定可以重用变量。

要修复代码，您需要在应共享变量的层中使用相同的名称。在documentation中指出：

reuse: Boolean, whether to reuse the weights of a previous layer by the same name.

此外，要调试代码并确保var指向同一位置，您只需删除reuse参数，并确保在尝试运行模型时出现异常。

Tensorflow的variable_scope（）和tf.AUTO_REUSE将不会在for循环中重用变量

2 个答案: