当前,如果有人在TensorFlow中定义了一个包含BatchNorm或LRN的网络,并尝试添加一个损失项,该损失项包含了相对于输入的交叉熵的梯度,则在以下几行将引发错误:
LookupError: No gradient defined for operation 'clone_0/gradients/clone_0/InceptionV3/InceptionV3/Conv2d_1a_3x3/BatchNorm/FusedBatchNorm_grad/FusedBatchNormGrad' (op type: FusedBatchNormGrad)
如果我们在网络中有一个tf.nn.lrn层,则会引发类似的情况:
LookupError: gradient registry has no entry for: LRNGrad
During handling of the above exception, another exception occurred:
...................
LookupError: No gradient defined for operation 'gradients/norm1_grad/LRNGrad' (op type: LRNGrad).
链接到LRN的github问题-https://github.com/tensorflow/tensorflow/issues/22825
对于tf.nn.sparse_softmax_cross_entropy_with_logits操作,也缺少二阶梯度,并且出现以下错误:
LookupError: Gradient explicitly disabled. Reason: b"Currently there is no way to take the second derivative of sparse_softmax_cross_entropy_with_logits due to the fused implementation's interaction with tf.gradients()" .
链接到sparse_softmax_cross_entropy_with_logits的github问题-https://github.com/tensorflow/tensorflow/issues/11626
这些错误表明,融合操作不支持高阶导数。我假设所有张量流操作都支持高阶导数计算。不是吗?