Tensorflow:用于双向RNN的OutputProjectionWrapper

时间:2017-10-29 04:07:27

标签: tensorflow neural-network recurrent-neural-network

是否有可能(如果是这样,如何)将OutputProjectionWrapper与tensorflow中的双向rnn结合使用?对于香草单向RNN,机制很简单:

cells = []

cell1 = tf.contrib.rnn.LSTMCell(num_units = 64, activation = tf.nn.tanh)  
cell1 = tf.contrib.rnn.DropoutWrapper(cell1, output_keep_prob= dropout)
cell1 = tf.contrib.rnn.OutputProjectionWrapper(cell1, output_size=numOutputFeatures)

cells.append(cell1)
cell = tf.contrib.rnn.MultiRNNCell(cells)
outputs, state = tf.nn.dynamic_rnn(cell, source, dtype=tf.float32, sequence_length = lengths)

但是,将其转移到双向RNN并不明显:

  1. 如果我们制作一个前向和后向层,每个都有一个投影包装器,并将它们用作bidirectional_dynamic_rnn的输入,这显然是错误的构造 - 我们想要收集两者的输出将它们组合在一起并将它们全部投射出来
  2. 这看起来像这样:

    cellfw = tf.contrib.rnn.LSTMCell(num_units = 64, activation = tf.nn.tanh)  
    cellfw = tf.contrib.rnn.DropoutWrapper(cellfw, output_keep_prob= dropout)
    cellfw = tf.contrib.rnn.OutputProjectionWrapper(cellfw, output_size=numOutputFeatures)
    
    cellbw = tf.contrib.rnn.LSTMCell(num_units = 64, activation = tf.nn.tanh)  
    cellbw = tf.contrib.rnn.DropoutWrapper(cellbw, output_keep_prob= dropout)
    cellbw = tf.contrib.rnn.OutputProjectionWrapper(cellbw, output_size=numOutputFeatures)
    
    outputs, state = tf.nn.bidirectional_dynamic_rnn(cellfw = cellfw, cellbw = cellbw, inputs = source, dtype=tf.float32, sequence_length = lengths)
    
    1. bidirectional_dynamic_rnn的输出实际上是一个元组(事实上它将说明前一种方法的错误。)如果我们可以将投影包装器应用于组合单元格,那将是有意义的
    2. ...然而

      cellfw = tf.contrib.rnn.LSTMCell(num_units = 64, activation = tf.nn.tanh)  
      cellfw = tf.contrib.rnn.DropoutWrapper(cellfw, output_keep_prob= dropout)  
      cellbw = tf.contrib.rnn.LSTMCell(num_units = 64, activation = tf.nn.tanh)  
      cellbw = tf.contrib.rnn.DropoutWrapper(cellbw, output_keep_prob= dropout)
      

      ...我们不能在这里使用tf.concat加入单元格,因为单元格不是张量。

      outputs, state = tf.nn.bidirectional_dynamic_rnn(cellfw = cellfw, cellbw = cellbw, inputs = source, dtype=tf.float32, sequence_length = lengths)
      
      outputs_fw = outputs[0]
      outputs_bw = outputs[1]
      outputs_combined = tf.concat([outputs_fw, outputs_bw], axis=2)
      projected = tf.contrib.rnn.OutputProjectionWrapper(outputs_combined, output_size=numOutputFeatures)
      

      虽然我们可以在这里使用tf.concat,但我们仍然无法应用投影包装器,因为它需要一个单元而不是一个张量。

      如何做到这一点?

0 个答案:

没有答案