Question

我一直在使用张量流来研究LSTM的时间序列预测。现在，我想尝试序列序列（seq2seq）。在官方网站上有一个教程，显示嵌入的NMT。那么，如何在没有嵌入的情况下使用这个新的seq2seq模块呢？（直接使用时间序列“序列”）。

# 1. Encoder
encoder_cell = tf.contrib.rnn.BasicLSTMCell(LSTM_SIZE)
encoder_outputs, encoder_state = tf.nn.static_rnn(
  encoder_cell,
  x,
  dtype=tf.float32)

# Decoder
decoder_cell = tf.nn.rnn_cell.BasicLSTMCell(LSTM_SIZE)


helper = tf.contrib.seq2seq.TrainingHelper(
    decoder_emb_inp, decoder_lengths, time_major=True)


decoder = tf.contrib.seq2seq.BasicDecoder(
  decoder_cell, helper, encoder_state)

# Dynamic decoding
outputs, _ = tf.contrib.seq2seq.dynamic_decode(decoder)
outputs = outputs[-1]

# output is result of linear activation of last layer of RNN
weight = tf.Variable(tf.random_normal([LSTM_SIZE, N_OUTPUTS]))
bias = tf.Variable(tf.random_normal([N_OUTPUTS]))
predictions = tf.matmul(outputs, weight) + bias

如果我使用input_seq = x和output_seq = label，那么TrainingHelper（）的参数应该是什么？

decoder_emb_inp ??? decoder_lengths ???

其中input_seq是序列的前8个点，output_seq是序列的最后2个点。先谢谢你！

Answer 1

我使用非常基本的...square使其无需嵌入即可工作：

InferenceHelper

我的输入是inference_helper = tf.contrib.seq2seq.InferenceHelper( sample_fn=lambda outputs: outputs, sample_shape=[dim], sample_dtype=dtypes.float32, start_inputs=start_tokens, end_fn=lambda sample_ids: False)形状的浮点数。对于下面的示例，[batch_size, time, dim]将为1，但是可以很容易地扩展到更大的尺寸。这是代码的相关部分：

dim

看看this question。我也发现这个tutorial对于理解seq2seq模型非常有用，尽管它确实使用了嵌入。因此，像我上面发布的那样，用projection_layer = tf.layers.Dense( units=1, # = dim kernel_initializer=tf.truncated_normal_initializer( mean=0.0, stddev=0.1)) # Training Decoder training_decoder_output = None with tf.variable_scope("decode"): # output_data doesn't exist during prediction phase. if output_data is not None: # Prepend the "go" token go_tokens = tf.constant(go_token, shape=[batch_size, 1, 1]) dec_input = tf.concat([go_tokens, target_data], axis=1) # Helper for the training process. training_helper = tf.contrib.seq2seq.TrainingHelper( inputs=dec_input, sequence_length=[output_size] * batch_size) # Basic decoder training_decoder = tf.contrib.seq2seq.BasicDecoder( dec_cell, training_helper, enc_state, projection_layer) # Perform dynamic decoding using the decoder training_decoder_output = tf.contrib.seq2seq.dynamic_decode( training_decoder, impute_finished=True, maximum_iterations=output_size)[0] # Inference Decoder # Reuses the same parameters trained by the training process. with tf.variable_scope("decode", reuse=tf.AUTO_REUSE): start_tokens = tf.constant( go_token, shape=[batch_size, 1]) # The sample_ids are the actual output in this case (not dealing with any logits here). # My end_fn is always False because I'm working with a generator that will stop giving # more data. You may extend the end_fn as you wish. E.g. you can append end_tokens # and make end_fn be true when the sample_id is the end token. inference_helper = tf.contrib.seq2seq.InferenceHelper( sample_fn=lambda outputs: outputs, sample_shape=[1], # again because dim=1 sample_dtype=dtypes.float32, start_inputs=start_tokens, end_fn=lambda sample_ids: False) # Basic decoder inference_decoder = tf.contrib.seq2seq.BasicDecoder(dec_cell, inference_helper, enc_state, projection_layer) # Perform dynamic decoding using the decoder inference_decoder_output = tf.contrib.seq2seq.dynamic_decode( inference_decoder, impute_finished=True, maximum_iterations=output_size)[0]代替他们的GreedyEmbeddingHelper。

P.s。我将完整的代码发布在https://github.com/Andreea-G/tensorflow_examples

如何在没有嵌入的情况下使用tensorflow seq2seq？

1 个答案: