Tensorflow版本1.0
我的问题是encoder_state
参数对tf.contrib.seq2seq attention_decoder_fn_train
期望的维度。
可以采用多层编码器状态输出吗?
上下文:
我想在 tensorflow 1.0 中创建多层双向注意力seq2seq 。
我的编码器:
cell = LSTM(n)
cell = MultiRnnCell([cell]*4)
((encoder_fw_outputs,encoder_bw_outputs),
(encoder_fw_state,encoder_bw_state)) = (tf.nn.bidirectional_dynamic_rnn(cell_fw=cell, cell_bw = cell.... )
现在,多层双向编码器为每一层返回编码器cell_states[c]
和hidden_states[h]
,并且还用于向后和向前传递。
我连接正向传递状态和反向传递状态以将其传递给encoder_state:
self.encoder_state = tf.concat((encoder_fw_state, encoder_bw_state), -1)
我把它传递给我的解码器:
decoder_fn_train = seq2seq.simple_decoder_fn_train(encoder_state=self.encoder_state)
(self.decoder_outputs_train,
self.decoder_state_train,
self.decoder_context_state_train) = seq2seq.dynamic_rnn_decoder(cell=decoder_cell,... )
但它会出现以下错误:
ValueError: The two structures don't have the same number of elements. First structure: Tensor("BidirectionalEncoder/transpose:0", shape=(?, 2, 2, 20), dtype=float32), second structure: (LSTMStateTuple(c=20, h=20), LSTMStateTuple(c=20, h=20)).
我的decoder_cell
也是多层次的。
1:
答案 0 :(得分:1)
我发现了我的实施问题。所以在这里发布。
问题是w.r.t.连接encoder_fw_state
和encoder_bw_state
。正确的方法如下:
self.encoder_state = []
for i in range(self.num_layers):
if isinstance(encoder_fw_state[i], LSTMStateTuple):
encoder_state_c = tf.concat((encoder_fw_state[i].c, encoder_bw_state[i].c), 1, name='bidirectional_concat_c')
encoder_state_h = tf.concat((encoder_fw_state[i].h, encoder_bw_state[i].h), 1, name='bidirectional_concat_h')
encoder_state = LSTMStateTuple(c=encoder_state_c, h=encoder_state_h)
elif isinstance(encoder_fw_state[i], tf.Tensor):
encoder_state = tf.concat((encoder_fw_state[i], encoder_bw_state[i]), 1, name='bidirectional_concat')
self.encoder_state.append(encoder_state)
self.encoder_state = tuple(self.encoder_state)