Tensorflow 1.0 Seq2Seq解码器功能

时间:2017-04-10 04:43:23

标签: python-2.7 tensorflow regression sequence-to-sequence

我试图为时间序列分析制作一个Seq2Seq回归示例,并且我使用了在Dev Summit上提供的Seq2Seq库,这是目前Tensorflow GitHub分支r1.0上的代码

我很难理解解码器功能如何适用于Seq2Seq,特别是对于" cell_output"。

据我所知,num_decoder_symbols是每个时间步骤要解码的类/字数。我让它在我可以进行训练的地方工作。但是,我不知道为什么我不能替换功能数量(num_features)而不是num_decoder_symbols。基本上,我希望能够在没有教师强制的情况下运行解码器,换句话说,将前一时间步的输出作为下一个时间步的输入。

 with ops.name_scope(name, "simple_decoder_fn_inference",
                    [time, cell_state, cell_input, cell_output,
                     context_state]):
  if cell_input is not None:
    raise ValueError("Expected cell_input to be None, but saw: %s" %
                     cell_input)
  if cell_output is None:
    # invariant that this is time == 0
    next_input_id = array_ops.ones([batch_size,], dtype=dtype) * (
        start_of_sequence_id)
    done = array_ops.zeros([batch_size,], dtype=dtypes.bool)
    cell_state = encoder_state
    cell_output = array_ops.zeros([num_decoder_symbols],
                                  dtype=dtypes.float32)

以下是原始代码的链接:https://github.com/tensorflow/tensorflow/blob/r1.0/tensorflow/contrib/seq2seq/python/ops/decoder_fn.py

为什么我不需要为单元格输出传递batch_size?

    cell_output = array_ops.zeros([batch_size, num_decoder_symbols],
                                  dtype=dtypes.float32)

当尝试使用此代码创建我自己的回归Seq2Seq示例时,我没有输出概率/类,而是有一个维数num_features的实值向量,而不是类概率数组。据我所知,我以为我可以用num_features替换num_decoder_symbols,如下所示:

     def decoder_fn(time, cell_state, cell_input, cell_output, context_state):
    """ 
    Again same as in simple_decoder_fn_inference but for regression on sequences with a fixed length
    """
    with ops.name_scope(name, "simple_decoder_fn_inference", [time, cell_state, cell_input, cell_output, context_state]):
        if cell_input is not None:
            raise ValueError("Expected cell_input to be None, but saw: %s" % cell_input)
        if cell_output is None:
            # invariant that this is time == 0
            next_input = array_ops.ones([batch_size, num_features], dtype=dtype)
            done = array_ops.zeros([batch_size], dtype=dtypes.bool)
            cell_state = encoder_state
            cell_output = array_ops.zeros([num_features], dtype=dtypes.float32)
        else:
            cell_output = output_fn(cell_output)
            done = math_ops.equal(0,1) # hardcoded hack just to properly define done
        next_input = cell_output
        # if time > maxlen, return all true vector
        done = control_flow_ops.cond(math_ops.greater(time, maximum_length),
                                     lambda: array_ops.ones([batch_size,], dtype=dtypes.bool),
                                     lambda: done)
        return (done, cell_state, next_input, cell_output, context_state)
return decoder_fn

但是,我收到以下错误:

File "/opt/DL/tensorflow/lib/python2.7/site-packages/tensorflow/contrib/seq2seq/python/ops/seq2seq.py", line 212, in dynamic_rnn_decoder
swap_memory=swap_memory, scope=scope)
File "/opt/DL/tensorflow/lib/python2.7/site-packages/tensorflow/python/ops/rnn.py", line 1036, in raw_rnn
swap_memory=swap_memory)
File "/opt/DL/tensorflow/lib/python2.7/site-packages/tensorflow/python/ops/control_flow_ops.py", line 2605, in while_loop
result = context.BuildLoop(cond, body, loop_vars, shape_invariants)
File "/opt/DL/tensorflow/lib/python2.7/site-packages/tensorflow/python/ops/control_flow_ops.py", line 2438, in BuildLoop
pred, body, original_loop_vars, loop_vars, shape_invariants)
File "/opt/DL/tensorflow/lib/python2.7/site-packages/tensorflow/python/ops/control_flow_ops.py", line 2388, in _BuildLoop
body_result = body(*packed_vars_for_body)
File "/opt/DL/tensorflow/lib/python2.7/site-packages/tensorflow/python/ops/rnn.py", line 980, in body
(next_output, cell_state) = cell(current_input, state)
File "/opt/DL/tensorflow/lib/python2.7/site-packages/tensorflow/contrib/rnn/python/ops/core_rnn_cell_impl.py", line 327, in __call__
input_size = inputs.get_shape().with_rank(2)[1]
File "/opt/DL/tensorflow/lib/python2.7/site-packages/tensorflow/python/framework/tensor_shape.py", line 635, in with_rank
raise ValueError("Shape %s must have rank %d" % (self, rank))
ValueError: Shape (100,) must have rank 2

结果,我传递了这样的batch_size,以获得等级2的形状:

    cell_output = array_ops.zeros([batch_size, num_features],
                                  dtype=dtypes.float32)

但是我得到以下错误,其中Shape是等级3并且想要等级2:

File "/opt/DL/tensorflow/lib/python2.7/site-packages/tensorflow/contrib/seq2seq/python/ops/seq2seq.py", line 212, in dynamic_rnn_decoder
swap_memory=swap_memory, scope=scope)
File "/opt/DL/tensorflow/lib/python2.7/site-packages/tensorflow/python/ops/rnn.py", line 1036, in raw_rnn
swap_memory=swap_memory)
File "/opt/DL/tensorflow/lib/python2.7/site-packages/tensorflow/python/ops/control_flow_ops.py", line 2605, in while_loop
result = context.BuildLoop(cond, body, loop_vars, shape_invariants)
File "/opt/DL/tensorflow/lib/python2.7/site-packages/tensorflow/python/ops/control_flow_ops.py", line 2438, in BuildLoop
pred, body, original_loop_vars, loop_vars, shape_invariants)
File "/opt/DL/tensorflow/lib/python2.7/site-packages/tensorflow/python/ops/control_flow_ops.py", line 2388, in _BuildLoop
body_result = body(*packed_vars_for_body)
File "/opt/DL/tensorflow/lib/python2.7/site-packages/tensorflow/python/ops/rnn.py", line 980, in body
(next_output, cell_state) = cell(current_input, state)
File "/opt/DL/tensorflow/lib/python2.7/site-packages/tensorflow/contrib/rnn/python/ops/core_rnn_cell_impl.py", line 327, in __call__
input_size = inputs.get_shape().with_rank(2)[1]
File "/opt/DL/tensorflow/lib/python2.7/site-packages/tensorflow/python/framework/tensor_shape.py", line 635, in with_rank
raise ValueError("Shape %s must have rank %d" % (self, rank))
ValueError: Shape (10, 10, 100) must have rank 2

0 个答案:

没有答案