是否有可能(如果是这样,如何)将OutputProjectionWrapper与tensorflow中的双向rnn结合使用?对于香草单向RNN,机制很简单:
cells = []
cell1 = tf.contrib.rnn.LSTMCell(num_units = 64, activation = tf.nn.tanh)
cell1 = tf.contrib.rnn.DropoutWrapper(cell1, output_keep_prob= dropout)
cell1 = tf.contrib.rnn.OutputProjectionWrapper(cell1, output_size=numOutputFeatures)
cells.append(cell1)
cell = tf.contrib.rnn.MultiRNNCell(cells)
outputs, state = tf.nn.dynamic_rnn(cell, source, dtype=tf.float32, sequence_length = lengths)
但是,将其转移到双向RNN并不明显:
bidirectional_dynamic_rnn
的输入,这显然是错误的构造 - 我们想要收集两者的输出将它们组合在一起并将它们全部投射出来这看起来像这样:
cellfw = tf.contrib.rnn.LSTMCell(num_units = 64, activation = tf.nn.tanh)
cellfw = tf.contrib.rnn.DropoutWrapper(cellfw, output_keep_prob= dropout)
cellfw = tf.contrib.rnn.OutputProjectionWrapper(cellfw, output_size=numOutputFeatures)
cellbw = tf.contrib.rnn.LSTMCell(num_units = 64, activation = tf.nn.tanh)
cellbw = tf.contrib.rnn.DropoutWrapper(cellbw, output_keep_prob= dropout)
cellbw = tf.contrib.rnn.OutputProjectionWrapper(cellbw, output_size=numOutputFeatures)
outputs, state = tf.nn.bidirectional_dynamic_rnn(cellfw = cellfw, cellbw = cellbw, inputs = source, dtype=tf.float32, sequence_length = lengths)
bidirectional_dynamic_rnn
的输出实际上是一个元组(事实上它将说明前一种方法的错误。)如果我们可以将投影包装器应用于组合单元格,那将是有意义的...然而
cellfw = tf.contrib.rnn.LSTMCell(num_units = 64, activation = tf.nn.tanh)
cellfw = tf.contrib.rnn.DropoutWrapper(cellfw, output_keep_prob= dropout)
cellbw = tf.contrib.rnn.LSTMCell(num_units = 64, activation = tf.nn.tanh)
cellbw = tf.contrib.rnn.DropoutWrapper(cellbw, output_keep_prob= dropout)
...我们不能在这里使用tf.concat加入单元格,因为单元格不是张量。
outputs, state = tf.nn.bidirectional_dynamic_rnn(cellfw = cellfw, cellbw = cellbw, inputs = source, dtype=tf.float32, sequence_length = lengths)
outputs_fw = outputs[0]
outputs_bw = outputs[1]
outputs_combined = tf.concat([outputs_fw, outputs_bw], axis=2)
projected = tf.contrib.rnn.OutputProjectionWrapper(outputs_combined, output_size=numOutputFeatures)
虽然我们可以在这里使用tf.concat,但我们仍然无法应用投影包装器,因为它需要一个单元而不是一个张量。
如何做到这一点?