Question

我正在尝试使用tf.contrib.seq2seq模块对某些数据进行预测（只是float32向量），但我使用TensorFlow中的seq2seq模块找到的所有示例都用于翻译，因此也用于嵌入。

我很难理解tf.contrib.seq2seq.Helper正在为Seq2Seq架构做些什么以及如何在我的情况下使用CustomHelper。

这就是我现在所做的：

import tensorflow as tf 
from tensorflow.python.layers import core as layers_core

input_seq_len = 15 # Sequence length as input
input_dim = 1 # Nb of features in input

output_seq_len = forecast_len = 20 # horizon length for forecasting
output_dim = 1 # nb of features to forecast


encoder_units = 200 # nb of units in each cell for the encoder
decoder_units = 200 # nb of units in each cell for the decoder

attention_units = 100

batch_size = 8


graph = tf.Graph()
with graph.as_default():

    learning_ = tf.placeholder(tf.float32)

    with tf.variable_scope('Seq2Seq'):

        # Placeholder for encoder input
        enc_input = tf.placeholder(tf.float32, [None, input_seq_len, input_dim])

        # Placeholder for decoder output - Targets
        target = tf.placeholder(tf.float32, [None, output_seq_len, output_dim])


        ### BUILD THE ENCODER

        # Build RNN cell
        encoder_cell = tf.nn.rnn_cell.BasicLSTMCell(encoder_units)

        initial_state = encoder_cell.zero_state(batch_size, dtype=tf.float32)

        # Run Dynamic RNN
        #   encoder_outputs: [batch_size, seq_size, num_units]
        #   encoder_state: [batch_size, num_units]
        encoder_outputs, encoder_state = tf.nn.dynamic_rnn(encoder_cell, enc_input, initial_state=initial_state)

        ## Attention layer

        attention_mechanism_bahdanau = tf.contrib.seq2seq.BahdanauAttention(
            num_units = attention_units, # depth of query mechanism
            memory = encoder_outputs, # hidden states to attend (output of RNN)
            normalize=False, # normalize energy term
            name='BahdanauAttention')

        attention_mechanism_luong = tf.contrib.seq2seq.LuongAttention(
            num_units = encoder_units,
            memory = encoder_outputs,
            scale=False,
            name='LuongAttention'
        )


        ### BUILD THE DECODER

        # Simple Dense layer to project from rnn_dim to the desired output_dim
        projection = layers_core.Dense(output_dim, use_bias=True, name="output_projection")

        helper = tf.contrib.seq2seq.TrainingHelper(target, sequence_length=[output_seq_len for _ in range(batch_size)])
 ## This is where I don't really know what to do in my case, is this function changing my data into [ GO, data, END] ?

        decoder_cell = tf.nn.rnn_cell.BasicLSTMCell(decoder_units)

        attention_cell = tf.contrib.seq2seq.AttentionWrapper(
            cell = decoder_cell,
            attention_mechanism = attention_mechanism_luong, # Instance of AttentionMechanism
            attention_layer_size = attention_units,
            name="attention_wrapper")

        initial_state = attention_cell.zero_state(batch_size=batch_size, dtype=tf.float32)
        initial_state = initial_state.clone(cell_state=encoder_state)

        decoder = tf.contrib.seq2seq.BasicDecoder(attention_cell, initial_state=initial_state, helper=helper, output_layer=projection)

        outputs, _, _ = tf.contrib.seq2seq.dynamic_decode(decoder=decoder)


        # Loss function:

        loss = 0.5*tf.reduce_sum(tf.square(outputs[0] - target), -1)
        loss = tf.reduce_mean(loss, 1)
        loss = tf.reduce_mean(loss)

        # Optimizer

        optimizer = tf.train.AdamOptimizer(learning_).minimize(loss)

我知道训练状态和推理状态对于Seq2seq架构是完全不同的，但我不知道如何使用模块中的助手来区分两者。我正在使用此模块，因为它对注意图层非常有用。如何使用Helper为解码器创建['Go'，[input_sequence]]？

如何使用tf.contrib.seq2seq.Helper进行非嵌入数据？

0 个答案: