使用tensorflow tf.control_dependencies和tf.layers.batch_normalization获取错误

时间:2018-11-27 16:18:23

标签: python tensorflow

我正在尝试构建具有两个损失函数的神经网络,这些函数像加权总和一样组合在一起。第一个简单地计算mean square error的稠密层和给定标签的线性输出,而另一个则大量使用嵌套的tf.map_fntf.layers.batch_normalization()使用了批处理规范层,因此我不得不将这些行添加到优化目标中:

    with tf.name_scope("Optimizer"):
        with tf.control_dependencies(tf.get_collection(tf.GraphKeys.UPDATE_OPS)):
            adam = tf.train.AdamOptimizer()
            self.train_op = adam.minimize(self.total_loss)

但是我得到了错误:

   AttributeError: 'NoneType' object has no attribute 'op'

它来自minimize()方法。如果删除控件依赖项,则没有错误。另外,如果我删除了依赖循环的第二个优化目标,则没有错误。我已经测试了正向传播中的第二次损失函数,它工作得很好。

任何想法如何跟踪问题?完整的错误日志:

Traceback (most recent call last):
  File "/home/mtarasov/anaconda3/lib/python3.6/site-packages/IPython/core/interactiveshell.py", line 3267, in run_code
    exec(code_obj, self.user_global_ns, self.user_ns)
  File "<ipython-input-2-6d5efdb6d091>", line 1, in <module>
    runfile('/home/mtarasov/PycharmProjects/ML/src/utils/model.py', wdir='/home/mtarasov/PycharmProjects/ML/src/utils')
  File "/home/mtarasov/Installations/pycharm-2018.2.4/helpers/pydev/_pydev_bundle/pydev_umd.py", line 198, in runfile
    pydev_imports.execfile(filename, global_vars, local_vars)  # execute the script
  File "/home/mtarasov/Installations/pycharm-2018.2.4/helpers/pydev/_pydev_imps/_pydev_execfile.py", line 18, in execfile
    exec(compile(contents+"\n", file, 'exec'), glob, loc)
  File "/home/mtarasov/PycharmProjects/ML/src/utils/model.py", line 168, in <module>
    model = Model().build()
  File "/home/mtarasov/PycharmProjects/ML/src/utils/model.py", line 60, in build
    self.train_op = adam.minimize(self.total_loss)
  File "/home/mtarasov/anaconda3/lib/python3.6/site-packages/tensorflow/python/training/optimizer.py", line 400, in minimize
    grad_loss=grad_loss)
  File "/home/mtarasov/anaconda3/lib/python3.6/site-packages/tensorflow/python/training/optimizer.py", line 514, in compute_gradients
    colocate_gradients_with_ops=colocate_gradients_with_ops)
  File "/home/mtarasov/anaconda3/lib/python3.6/site-packages/tensorflow/python/ops/gradients_impl.py", line 596, in gradients
    gate_gradients, aggregation_method, stop_gradients)
  File "/home/mtarasov/anaconda3/lib/python3.6/site-packages/tensorflow/python/ops/gradients_impl.py", line 663, in _GradientsHelper
    to_ops, from_ops, colocate_gradients_with_ops, func_graphs, xs)
  File "/home/mtarasov/anaconda3/lib/python3.6/site-packages/tensorflow/python/ops/gradients_impl.py", line 190, in _PendingCount
    between_op_list, between_ops, colocate_gradients_with_ops)
  File "/home/mtarasov/anaconda3/lib/python3.6/site-packages/tensorflow/python/ops/control_flow_ops.py", line 1432, in MaybeCreateControlFlowState
    loop_state.AddWhileContext(op, between_op_list, between_ops)
  File "/home/mtarasov/anaconda3/lib/python3.6/site-packages/tensorflow/python/ops/control_flow_ops.py", line 1244, in AddWhileContext
    grad_state = GradLoopState(forward_ctxt, outer_grad_state)
  File "/home/mtarasov/anaconda3/lib/python3.6/site-packages/tensorflow/python/ops/control_flow_ops.py", line 846, in __init__
    real_cnt, outer_grad_state)
  File "/home/mtarasov/anaconda3/lib/python3.6/site-packages/tensorflow/python/ops/control_flow_ops.py", line 2585, in AddBackpropLoopCounter
    name="b_count")
  File "/home/mtarasov/anaconda3/lib/python3.6/site-packages/tensorflow/python/ops/control_flow_ops.py", line 248, in _Enter
    data, frame_name, is_constant, parallel_iterations, name=name)
  File "/home/mtarasov/anaconda3/lib/python3.6/site-packages/tensorflow/python/ops/gen_control_flow_ops.py", line 178, in enter
    parallel_iterations=parallel_iterations, name=name)
  File "/home/mtarasov/anaconda3/lib/python3.6/site-packages/tensorflow/python/framework/op_def_library.py", line 787, in _apply_op_helper
    op_def=op_def)
  File "/home/mtarasov/anaconda3/lib/python3.6/site-packages/tensorflow/python/util/deprecation.py", line 454, in new_func
    return func(*args, **kwargs)
  File "/home/mtarasov/anaconda3/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 3155, in create_op
    op_def=op_def)
  File "/home/mtarasov/anaconda3/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 1746, in __init__
    self._control_flow_post_processing()
  File "/home/mtarasov/anaconda3/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 1757, in _control_flow_post_processing
    self._control_flow_context.AddOp(self)
  File "/home/mtarasov/anaconda3/lib/python3.6/site-packages/tensorflow/python/ops/control_flow_ops.py", line 2438, in AddOp
    self._AddOpInternal(op)
  File "/home/mtarasov/anaconda3/lib/python3.6/site-packages/tensorflow/python/ops/control_flow_ops.py", line 2475, in _AddOpInternal
    for x in external_inputs if x.outputs]
  File "/home/mtarasov/anaconda3/lib/python3.6/site-packages/tensorflow/python/ops/control_flow_ops.py", line 2475, in <listcomp>
    for x in external_inputs if x.outputs]
  File "/home/mtarasov/anaconda3/lib/python3.6/site-packages/tensorflow/python/ops/array_ops.py", line 80, in identity
    return gen_array_ops.identity(input, name=name)
  File "/home/mtarasov/anaconda3/lib/python3.6/site-packages/tensorflow/python/ops/gen_array_ops.py", line 3264, in identity
    "Identity", input=input, name=name)
  File "/home/mtarasov/anaconda3/lib/python3.6/site-packages/tensorflow/python/framework/op_def_library.py", line 787, in _apply_op_helper
    op_def=op_def)
  File "/home/mtarasov/anaconda3/lib/python3.6/site-packages/tensorflow/python/util/deprecation.py", line 454, in new_func
    return func(*args, **kwargs)
  File "/home/mtarasov/anaconda3/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 3155, in create_op
    op_def=op_def)
  File "/home/mtarasov/anaconda3/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 1746, in __init__
    self._control_flow_post_processing()
  File "/home/mtarasov/anaconda3/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 1757, in _control_flow_post_processing
    self._control_flow_context.AddOp(self)
  File "/home/mtarasov/anaconda3/lib/python3.6/site-packages/tensorflow/python/ops/control_flow_ops.py", line 2438, in AddOp
    self._AddOpInternal(op)
  File "/home/mtarasov/anaconda3/lib/python3.6/site-packages/tensorflow/python/ops/control_flow_ops.py", line 2466, in _AddOpInternal
    self._MaybeAddControlDependency(op)
  File "/home/mtarasov/anaconda3/lib/python3.6/site-packages/tensorflow/python/ops/control_flow_ops.py", line 2504, in _MaybeAddControlDependency
    op._add_control_input(self.GetControlPivot().op)
AttributeError: 'NoneType' object has no attribute 'op

2 个答案:

答案 0 :(得分:0)

我不知道到底发生了什么,但是我已将所有层(包括批处理规范)从tf.layers替换为tf.keras.layers,并且错误消失了!

答案 1 :(得分:0)

增加mcstarioni的答案。如前所述,用tf.keras.layers.BatchNormalization替换批处理规范层似乎可以消除该错误。但是,这是因为keras中的BatchNormalization并未像here那样在UPDATE_OPS中添加批处理规范参数,因为它使用了不同的训练方式。如果检查移动平均值和方差,您会注意到在训练过程中仅通过运行train_op并不会更新它们。 除了运行layer.update 之外,还必须运行train_op,这一点很重要。

或者,如果可能的话,请尝试删除嵌套的map_fn