张量流w.r.t隐藏层中输出的梯度

时间:2018-03-20 02:49:26

标签: python tensorflow gradient-descent

在张量流中,我试图计算相对于表示为layer_out 的隐藏层,表示为输出的n层NN模型的输出的梯度i = 1,2,...,n。

的任何一层

输出的形状是[128,10],layer_out的形状是[128,100],其中128是批量大小。我为此目的使用以下命令。

l_grad = tf.gradients(output, layer_out)[0] 

我期望的渐变形状是[10,100]或[100,10]。假设梯度是整个批次的平均值。但是,我得到的形状是[128,100]

我尝试过的一个途径是在梯度计算之前使用tf.reduce_mean()来平均批次中的两个量。但是,结果是无列表,并显示以下错误

Traceback (most recent call last):
  File "Test_Gen_Error_MNIST_EDL.py", line 128, in <module>
    Results[i,4], Results[i,5], Results[i,6] = Analyse_custom_Optimizer_EDL_old(X_train,y_train,X_test,y_test, Kappa_s[i])
  File "Test_Gen_Error_MNIST_EDL.py", line 60, in Analyse_custom_Optimizer_EDL_old
    model = model.init_NN_custom(classes, 1e-03, depth, tf.sigmoid,'EDL')
  File "/usr/local/home/krm9c/Documents/Research/ErrorDriven-Learning/MN/Class_gen_Error.py", line 265, in init_NN_custom
    Error_Loss, self.classifier['class'], array, dec)
  File "/usr/local/home/krm9c/Documents/Research/ErrorDriven-Learning/MN/Class_gen_Error.py", line 192, in direct_feedback_alignement
    print_grad_shape = tf.Print(l_grad, [tf.shape(l_grad)], 'L_gradient shape')
  File "/usr/local/home/krm9c/anaconda2/envs/deep/lib/python2.7/site-packages/tensorflow/python/ops/array_ops.py", line 276, in shape
    return shape_internal(input, name, optimize=True, out_type=out_type)
  File "/usr/local/home/krm9c/anaconda2/envs/deep/lib/python2.7/site-packages/tensorflow/python/ops/array_ops.py", line 300, in shape_internal
    input_tensor = ops.convert_to_tensor(input)
  File "/usr/local/home/krm9c/anaconda2/envs/deep/lib/python2.7/site-packages/tensorflow/python/framework/ops.py", line 932, in convert_to_tensor
    as_ref=False)
  File "/usr/local/home/krm9c/anaconda2/envs/deep/lib/python2.7/site-packages/tensorflow/python/framework/ops.py", line 1022, in internal_convert_to_tensor
    ret = conversion_func(value, dtype=dtype, name=name, as_ref=as_ref)
  File "/usr/local/home/krm9c/anaconda2/envs/deep/lib/python2.7/site-packages/tensorflow/python/framework/constant_op.py", line 233, in _constant_tensor_conversion_function
    return constant(v, dtype=dtype, name=name)
  File "/usr/local/home/krm9c/anaconda2/envs/deep/lib/python2.7/site-packages/tensorflow/python/framework/constant_op.py", line 212, in constant
    value, dtype=dtype, shape=shape, verify_shape=verify_shape))
  File "/usr/local/home/krm9c/anaconda2/envs/deep/lib/python2.7/site-packages/tensorflow/python/framework/tensor_util.py", line 401, in make_tensor_proto
    raise ValueError("None values not supported.")
ValueError: None values not supported.

我在这里遗漏了一些东西,但我无法弄明白。任何方向都非常感谢。 谢谢

0 个答案:

没有答案