Question

我正在处理可变长度序列数据。我试图实现两个LSTM体系结构。目的是第二个LSTM应该使用第一个LSTM的最后一个有效状态（忽略填充）作为初始状态。初始状态由cell_state和hidden_state组成。但是

keras.layers.LSTM(num_units, return_state=True)

仅返回最后一个hidden_state和最后一个cell_state。我想要的是每个序列的最后一个有效states。我可以使用Masking，但不能与CuDNNLSTM一起使用。

我发现solution在Keras中的每个时间步都提取了cell states。但是我无法在Tensorflow中复制它。乏味的while_loop杀死了我。我遇到错误：

ValueError: Initializer for variable while_18/lstm_6/kernel/ is from inside a control-flow construct, such as a loop or conditional. When creating a variable inside a loop or conditional, use a lambda as the initializer.

在下面您可以找到我的尝试：

import keras
from keras.layers import Input, LSTM, Lambda
import tensorflow as tf

maxlen = 10
input_dim = 10
units = 5

inputs = Input((maxlen, input_dim), dtype = tf.float32)

rnn = LSTM(units, return_state=True)

def get_indexer(t):
    return Lambda(lambda x, t: x[:, t, :], arguments={'t':t}, output_shape=lambda s: (s[0], s[2]))

def expand(x):
    return keras.backend.expand_dims(x, 1)

expand_layer = Lambda(expand, output_shape=lambda s: (s[0], 1, s[1]))
state = tf.Variable(tf.zeros([10]))
states = tf.TensorArray(dtype=tf.float32, size=0, dynamic_size=True, name='states')
iters = tf.constant(10, name='iters')

def cond(i, iters, states):
    return tf.less(i, iters)

def body(i, iters, states):
    input_t = get_indexer(i)(inputs)  # basically input_t = inputs[:, t, :]
    input_t = expand_layer(input_t)
    output_t, h, c = rnn(input_t, initial_state=state)
    temp_state = h, c
    assign_op = tf.assign(state, temp_state)
    states = states.write(step, state)
    return states

states = tf.while_loop(cond, body, [0, iters, states])

在张量流的每个时间步提取单元状态

0 个答案: