使用AttentionCellWrapper导致的类型错误:无法迭代张量

时间:2019-03-01 14:18:28

标签: python python-2.7 tensorflow attention-model

系统信息 -操作系统平台和发行版:Linux Manjaro Illyria 18.0.3 -移动设备:如果问题发生在移动设备上:与移动设备无关 -从以下位置安装TensorFlow:二进制 -TensorFlow版本:('v1.12.0-0-ga6d8ffae09','1.12.0') -Python版本:2.7.15 -CUDA / cuDNN版本:无CUDA -GPU模型和内存:无计算GPU

描述当前行为 我正在尝试将tf.contrib.rnn.AttentionCellWrapper添加到tf.nn.rnn_cell.GRUCell。在添加包装器之前,程序将完全正常运行。添加包装器后,程序将报告错误,显示在下面的回溯中。

错误Tensor objects are only iterable when eager execution is enabled使我感到困惑,因为我尝试启用急切的执行,并且结果相同。 我已将GRUCell更改为LSMMCell或其他与RNN相关的Cell,结果是相同的。 我尝试了多个tensorflow版本,结果是相同的。 我正在修改的代码与this code相同,而我正在尝试的是在126至127行之间添加AttentionCellWrapper。

描述预期的行为 该程序应该运行正常,没有任何错误。与普通代码相比,我所做的所有更改都是添加了AttentionCellWrapper。 官方文档也没有解释此错误的原因。

重现问题的代码 如上所述,带有AttentionCellWrapper的常规代码以及触发错误的函数如下。完整的代码还涵盖了repo中的其他文件。

def build_model(self):

        self.X = tf.placeholder(tf.int32, [self.batch_size], name='input')
        self.Y = tf.placeholder(tf.int32, [self.batch_size], name='output')
        self.state = [tf.placeholder(tf.float32, [self.batch_size, self.rnn_size], name='rnn_state') for _ in xrange(self.layers)]
        self.global_step = tf.Variable(0, name='global_step', trainable=False)

        with tf.variable_scope('gru_layer'):
            sigma = self.sigma if self.sigma != 0 else np.sqrt(6.0 / (self.n_items + self.rnn_size))
            if self.init_as_normal:
                initializer = tf.random_normal_initializer(mean=0, stddev=sigma)
            else:
                initializer = tf.random_uniform_initializer(minval=-sigma, maxval=sigma)
            embedding = tf.get_variable('embedding', [self.n_items, self.rnn_size], initializer=initializer)
            softmax_W = tf.get_variable('softmax_w', [self.n_items, self.rnn_size], initializer=initializer)
            softmax_b = tf.get_variable('softmax_b', [self.n_items], initializer=tf.constant_initializer(0.0))

            cell = rnn_cell.GRUCell(self.rnn_size, activation=self.hidden_act)
            attn_cell = tf.contrib.rnn.AttentionCellWrapper(cell, attn_length=20)  # state_is_tuple is True by default
            drop_cell = rnn_cell.DropoutWrapper(attn_cell, output_keep_prob=self.dropout_p_hidden)
            stacked_cell = rnn_cell.MultiRNNCell([drop_cell] * self.layers)

            inputs = tf.nn.embedding_lookup(embedding, self.X)
            output, state = stacked_cell(inputs, tuple(self.state))
            self.final_state = state

        if self.is_training:
            '''
            Use other examples of the minibatch as negative samples.
            '''
            sampled_W = tf.nn.embedding_lookup(softmax_W, self.Y)
            sampled_b = tf.nn.embedding_lookup(softmax_b, self.Y)
            logits = tf.matmul(output, sampled_W, transpose_b=True) + sampled_b
            self.yhat = self.final_activation(logits)
            self.cost = self.loss_function(self.yhat)
        else:
            logits = tf.matmul(output, softmax_W, transpose_b=True) + softmax_b
            self.yhat = self.final_activation(logits)

        if not self.is_training:
            return

        self.lr = tf.maximum(1e-5,tf.train.exponential_decay(self.learning_rate, self.global_step, self.decay_steps, self.decay, staircase=True)) 

        '''
        Try different optimizers.
        '''
        #optimizer = tf.train.AdagradOptimizer(self.lr)
        optimizer = tf.train.AdamOptimizer(self.lr)
        #optimizer = tf.train.AdadeltaOptimizer(self.lr)
        #optimizer = tf.train.RMSPropOptimizer(self.lr)

        tvars = tf.trainable_variables()
        gvs = optimizer.compute_gradients(self.cost, tvars)
        if self.grad_cap > 0:
            capped_gvs = [(tf.clip_by_norm(grad, self.grad_cap), var) for grad, var in gvs]
        else:
            capped_gvs = gvs 
        self.train_op = optimizer.apply_gradients(capped_gvs, global_step=self.global_step)

其他信息/日志 包括任何有助于诊断问题的日志或源代码。如果包括回溯,请包括完整的回溯。大的日志和文件应该附加。

Traceback (most recent call last):
  File "/home/syh/.local/share/JetBrains/Toolbox/apps/PyCharm-P/ch-0/183.5429.31/helpers/pydev/pydevd.py", line 1741, in <module>
    main()
  File "/home/syh/.local/share/JetBrains/Toolbox/apps/PyCharm-P/ch-0/183.5429.31/helpers/pydev/pydevd.py", line 1735, in main
    globals = debugger.run(setup['file'], None, None, is_module)
  File "/home/syh/.local/share/JetBrains/Toolbox/apps/PyCharm-P/ch-0/183.5429.31/helpers/pydev/pydevd.py", line 1135, in run
    pydev_imports.execfile(file, globals, locals)  # execute the script
  File "/run/media/syh/0D11080D0D11080D/workspace/Recommender-System/exp/GRU4Rec_TensorFlow-master/main.py", line 86, in <module>
    gru4rec = model.GRU4Rec(sess, args)
  File "/run/media/syh/0D11080D0D11080D/workspace/Recommender-System/exp/GRU4Rec_TensorFlow-master/model.py", line 72, in __init__
    self.build_model()
  File "/run/media/syh/0D11080D0D11080D/workspace/Recommender-System/exp/GRU4Rec_TensorFlow-master/model.py", line 142, in build_model
    output, state = multi_cell(inputs, tuple(self.state))
  File "/run/media/syh/0D11080D0D11080D/workspace/Recommender-System/exp/venv/lib/python2.7/site-packages/tensorflow/python/ops/rnn_cell_impl.py", line 233, in __call__
    return super(RNNCell, self).__call__(inputs, state)
  File "/run/media/syh/0D11080D0D11080D/workspace/Recommender-System/exp/venv/lib/python2.7/site-packages/tensorflow/python/layers/base.py", line 374, in __call__
    outputs = super(Layer, self).__call__(inputs, *args, **kwargs)
  File "/run/media/syh/0D11080D0D11080D/workspace/Recommender-System/exp/venv/lib/python2.7/site-packages/tensorflow/python/keras/engine/base_layer.py", line 757, in __call__
    outputs = self.call(inputs, *args, **kwargs)
  File "/run/media/syh/0D11080D0D11080D/workspace/Recommender-System/exp/venv/lib/python2.7/site-packages/tensorflow/python/ops/rnn_cell_impl.py", line 1486, in call
    cur_inp, new_state = cell(cur_inp, cur_state)
  File "/run/media/syh/0D11080D0D11080D/workspace/Recommender-System/exp/venv/lib/python2.7/site-packages/tensorflow/python/ops/rnn_cell_impl.py", line 1282, in __call__
    output, new_state = self._cell(inputs, state, scope=scope)
  File "/run/media/syh/0D11080D0D11080D/workspace/Recommender-System/exp/venv/lib/python2.7/site-packages/tensorflow/python/ops/rnn_cell_impl.py", line 233, in __call__
    return super(RNNCell, self).__call__(inputs, state)
  File "/run/media/syh/0D11080D0D11080D/workspace/Recommender-System/exp/venv/lib/python2.7/site-packages/tensorflow/python/layers/base.py", line 374, in __call__
    outputs = super(Layer, self).__call__(inputs, *args, **kwargs)
  File "/run/media/syh/0D11080D0D11080D/workspace/Recommender-System/exp/venv/lib/python2.7/site-packages/tensorflow/python/keras/engine/base_layer.py", line 757, in __call__
    outputs = self.call(inputs, *args, **kwargs)
  File "/run/media/syh/0D11080D0D11080D/workspace/Recommender-System/exp/venv/lib/python2.7/site-packages/tensorflow/contrib/rnn/python/ops/rnn_cell.py", line 1190, in call
    state, attns, attn_states = state
  File "/run/media/syh/0D11080D0D11080D/workspace/Recommender-System/exp/venv/lib/python2.7/site-packages/tensorflow/python/framework/ops.py", line 459, in __iter__
    "Tensor objects are only iterable when eager execution is "
TypeError: Tensor objects are only iterable when eager execution is enabled. To iterate over this tensor use tf.map_fn.

state<type 'tuple'>: (<tf.Tensor 'rnn_state:0' shape=(50, 100) dtype=float32>,) __len__ = 1

state由该语句output, state = multi_cell(inputs, tuple(self.state))返回 哪里 self.state = [tf.placeholder(tf.float32, [self.batch_size, self.rnn_size], name='rnn_state') for _ in xrange(self.layers)] multi_cell = rnn_cell.MultiRNNCell([cell] * self.layers) inputs = tf.nn.embedding_lookup(embedding, self.X)

stateclass RNNCelltensorflow/python/ops/rnn_cell_impl.py中创建 但是,我看不到state是通过调试器由类RNNCell中的哪个语句创建的。

将tensorflow版本更新为1.13.1后,此问题仍然存在。

0 个答案:

没有答案