Question

我正在尝试使用Tensorflow构建生成RNN。我有一个预处理的数据集，它是sequence_length x 2048 x 2 numpy数组的列表。序列具有不同的长度。我一直在查看示例和文档，但我真的无法理解，例如，key是什么，或者我应该如何创建input_sequences字典等。

那么如何格式化numpy数组列表，每个数组代表一个排名n（本例中为2）张量的序列，以便能够使用这个batch_sequences_with_states方法？

Answer 1

玩具实施

我试过这个，我很乐意与你分享我的发现。这是一个玩具的例子。我尝试创建一个有效的示例，并观察输出的变化情况。特别是我使用了lstm的案例研究。对您而言，您可以定义转发网络。随意添加更多输入并像往常一样进行调整并按照文档进行操作。 https://www.tensorflow.org/versions/r0.11/api_docs/python/contrib.training/splitting_sequence_inputs_into_minibatches_with_state_saving#batch_sequences_with_states 我尝试了其他更微妙的例子，但我保留这个简单的版本来展示操作如何有用。特别是在字典中添加更多元素（输入序列和上下文序列）并观察变化。

两种方法

基本上我会使用两种方法：

tf.contrib.training.batch_sequences_with_states
tf.train.batch（）

我将从第一个开始，因为它将直接有用，然后我将展示如何用train.batch解决类似的问题。

我将基本上生成玩具numpy数组和张量，并用它来测试操作

import tensorflow as tf
batch_size = 32
num_unroll = 20
num_enqueue_threads = 20
lstm_size = 8
cell = tf.contrib.rnn.BasicLSTMCell(num_units=lstm_size)


#state size
state_size = cell.state_size[0];
initial_state_values = tf.zeros((state_size,), dtype=tf.float32)

# Initial states
initial_state_values = tf.zeros((state_size,), dtype=tf.float32)
initial_states = {"lstm_state": initial_state_values}

# Key should be string
#I used x as input sequence and y as input context. So that the
# keys should be 2.
key = ["1","2"]

#Toy data for our sample
x = tf.range(0, 12, name="x")
y = tf.range(12,24,name="y")

# convert to float
#I converted to float so as not to raise type mismatch erroe
x=tf.to_float(x)
y=tf.to_float(y)

#the input sequence as dictionary
#This is needed according to the tensorflow doc

sequences = {"x": x }

#Context Input
context = {"batch1": y}

 # Train batch with sequence state
batch_new = tf.contrib.training.batch_sequences_with_states(
    input_key=key,
    input_sequences=sequences,
    input_context=context,
    initial_states=initial_states,
    num_unroll=num_unroll,
    batch_size=batch_size,
    input_length = None,
    pad = True,
    num_threads=num_enqueue_threads,
    capacity=batch_size * num_enqueue_threads * 2)

 # To test what we have got type and observe the output of
 # the following
 # In short once in ipython notebook
 # type batch_new.[press tab] to see all options
 batch_new.key
 batch_new.sequences

 #splitting of  input. This generate input per epoch
 inputs_by_time = tf.split(inputs, num_unroll)
 assert len(inputs_by_time) == num_unroll

 # Get lstm or conv net output
 lstm_output, _ = tf.contrib.rnn.static_state_saving_rnn(
           cell,
           inputs_by_time,
           state_saver=batch_new,
           state_name=("lstm_state","lstm_state"))

按常规创建图表和队列

带＃和*的部件可以进一步调整以适应要求。

 # Create the graph, etc.
 init_op = tf.global_variables_initializer()

 #Create a session for running operations in the Graph.
 sess = tf.Session()

 # Initialize the variables (like the epoch counter).
 sess.run(init_op)

 # Start input enqueue threads.
 coord = tf.train.Coordinator()
 threads = tf.train.start_queue_runners(sess=sess, coord=coord)

 # For the part below uncomment 
 #*those comments with asterics to do other operations
 #*try:
  #*    while not coord.should_stop():
         #*Run training steps or whatever
         #*sess.run(train_op) # uncomment to run other ops


 #*except tf.errors.OutOfRangeError:
         #print('Done training -- epoch limit reached')
 #*finally:
      # When done, ask the threads to stop.
 coord.request_stop()
 # Wait for threads to finish.
 coord.join(threads)
 sess.close()

第二种方法

你也可以用一种非常有趣的方式使用train.batch：

import tensorflow as tf

#[0, 1, 2, 3, 4 ,...]
x = tf.range(0, 11, name="x")

# A queue that outputs 0,1,2,3,..
# slice end is useful for dequeuing
slice_end = 10
# instantiate variable y 
y = tf.slice(x, [0], [slice_end], name="y")

# Reshape y
y = tf.reshape(y,[10,1])
y=tf.to_float(y, name='ToFloat')

重要注意使用dynamic并使用padding将许多内容排入队列。随意玩两种选择。并比较输出！

batched_data = tf.train.batch( tensors=[y], batch_size=10, dynamic_pad=True, #enqueue_many=True, name="y_batch" ) batch_size = 128 ; lstm_cell = tf.contrib.rnn.LSTMCell(batch_size,forget_bias=1,state_is_tuple=True) val, state = tf.nn.dynamic_rnn(lstm_cell, batched_data, dtype=tf.float32)

<强>结论

目的是表明通过简单的例子我们可以深入了解操作细节。在您的情况下，您可以将其调整为卷积网络。

希望这有帮助！

如何使用Tensorflow的batch_sequences_with_states实用程序

1 个答案: