Question

当我尝试对相同输入进行重复卷积时，梯度变为零，重复次数超过1次。这可能会出错？

W = tf.Variable(tf.zeros([3, 3, 1, 1]))
output = input_image # a 4D tensor [batch_size, 16, 16, 1]
for _ in range(4):
    output = tf.nn.conv2d(
        output, 
        W, 
        [1, 2, 2, 1], 
        padding="SAME"
    )

preds = tf.reshape(output, shape=[batch_size])

loss = tf.reduce_mean(preds, labels)

# this gradient zero when num of repetitive layers > 1??
tf_gradient = tf.concat(0, tf.gradients(loss, W))
gradient = session.run(tf_gradient)

print(gradient.reshape(3**2))
#prints [ 0.  0.  0.  0.  0.  0.  0.  0.  0.]

Answer 1

对W使用随机初始化。像W = tf.get_variable(name="W", initializer=tf.glorot_uniform_initializer, shape=[3, 3, 1, 1], dtype=tf.float32)

这样的东西

重复卷积张量流的零梯度？

1 个答案: