Question

我想在tensorflow中使用这种渐变剪切技术。所以，我有下一个功能：

def create_optimizers(cost, collections, learning_rate):
'''
    Create optimizer for collections with associate learning rate
    Args:
      cost: Euclid cost functions
      collections: training collections
      learning_rate: learning rate for gradient descent
    Return:
      Gradient functions associate with tenosrflow collections
'''
optimizer = tf.train.MomentumOptimizer(learning_rate=learning_rate, momentum=0.9)

gradients = {}

sigma = 0.01
clip_val = tf.div(sigma, learning_rate)
#clip_val = 0.1
# clip gradient
for collection in collections:
    tvars = tf.get_collection(collection)

    grads, _ = tf.clip_by_global_norm(tf.gradients(cost, tvars), clip_norm=clip_val)
    train_op_wc = optimizer.apply_gradients(zip(grads, tvars))

return gradients

现在，它完成了常量限幅规范。但是当我用占位符填充它时，如：

weight_learning_rate = tf.placeholder(tf.float32, shape=[])
weight_optimizers = sp_net_util.create_optimizers(cost, weights_coll, weight_learning_rate)

将其作为

运行

sess.run(weight_optimizers[i],feed_dict={weight_learning_rate: private_weight_learning_rate,...})

我收到了这个错误：

  File ".../sp_train.py", line 70, in train
    weight_optimizers = sp_net_util.create_optimizers(cost, weights_coll, weight_learning_rate)
  File "".../sp_net_util.py", line 218, in create_optimizers
    grads, _ = tf.clip_by_global_norm(tf.gradients(cost, tvars), clip_norm=clip_val)
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/ops/clip_ops.py", line 208, in clip_by_global_norm
    constant_op.constant(1.0 / clip_norm, dtype=use_norm.dtype))
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/constant_op.py", line 163, in constant
    tensor_util.make_tensor_proto(value, dtype=dtype, shape=shape))
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/tensor_util.py", line 353, in make_tensor_proto
    _AssertCompatible(values, dtype)
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/tensor_util.py", line 290, in _AssertCompatible
    (dtype.name, repr(mismatch), type(mismatch).__name__))
TypeError: Expected float32, got <tf.Tensor 'weight_optimizer/clip_by_global_norm/truediv_1:0' shape=() dtype=float32> of type 'Tensor' instead.

如果我把一切都放好了。

那么，有任何方法可以在tensorflow中使用占位符吗？

如何使用张量流进行可调梯度修剪

0 个答案: