Question

我有一个用于图像压缩的自动编码器，其中编码的张量具有以下形状：（batch_size，12，64，48）。

batch_size是一批中要送入的图像数， 12是最后一个编码器层的通道数，它具有 64x48宽度/高度。

我想将其输入到ConvLSTM2D层，我希望ConvLSTM2D的输出具有与ConvLSTM2D的输入相同的尺寸。

目的是查看视频序列上的图像重建，而不是数据集中的无序图像。

在自动编码器体系结构中的编码器/解码器之间放置ConvLSTM2d一直很困难，尤其是因为大多数示例都使用顺序API，而我想在Keras中使用功能性API。

我尝试重塑输入，但错误仍然存在

import tensorflow as tf
import tensorflow.keras.backend as K


def LSTM_layer(input):

    input = tf.keras.backend.expand_dims(input, axis=-1)
    lstm1 = tf.keras.layers.ConvLSTM2D(filters=12, kernel_size=(3, 3), strides=(1, 1), data_format="channels_first",
                                        input_shape=(None, 12, 64, 48), 
                                        padding='same', return_sequences=True)(input)

    return lstm1

def build_model(input_shape):

    #create an input with input_shape as the size
    input_ = tf.keras.Input(shape=input_shape, name="input_node")
    lstm_features = LSTM_layer(input_)

    model = tf.keras.Model(inputs=input_, outputs=[lstm_features])
    return model

def main():

    input_shape = (12, 64, 48) #this is the size of the tensor which is outputted by my encoder, with channels_first assumed
    model = build_model(input_shape)

if __name__ == '__main__':
    main()

不幸的是，这引发了此错误：

Traceback (most recent call last):
  File "lstm.py", line 29, in <module>
    main()
  File "lstm.py", line 26, in main
    model = build_model(input_shape)
  File "lstm.py", line 20, in build_model
    model = tf.keras.Model(inputs=input_, outputs=[lstm_features])
  File "/home/hallab/.local/lib/python3.5/site-packages/tensorflow/python/keras/engine/training.py", line 121, in __init__
    super(Model, self).__init__(*args, **kwargs)
  File "/home/hallab/.local/lib/python3.5/site-packages/tensorflow/python/keras/engine/network.py", line 80, in __init__
    self._init_graph_network(*args, **kwargs)
  File "/home/hallab/.local/lib/python3.5/site-packages/tensorflow/python/training/checkpointable/base.py", line 474, in _method_wrapper
    method(self, *args, **kwargs)
  File "/home/hallab/.local/lib/python3.5/site-packages/tensorflow/python/keras/engine/network.py", line 224, in _init_graph_network
    '(thus holding past layer metadata). Found: ' + str(x))
ValueError: Output tensors to a Model must be the output of a TensorFlow `Layer` (thus holding past layer metadata). Found: Tensor("conv_lst_m2d/transpose_1:0", shape=(?, 12, 12, 48, 1), dtype=float32)

有关此错误的大多数文章都指示将操作包装在lambda中。.但是我没有在此处实现自定义操作，这应该是keras TF层...对吧？

此外，在我的实现中，我希望LSTM单位的输出张量与输入的相同，我也可以得到一些反馈吗？

谢谢。

Answer 1

您可以使用Lambda包装输出形式K.expand_dims，然后再将其输入到下一层：

import tensorflow as tf
import tensorflow.keras.backend as K
from tensorflow.keras.layers import Lambda

def expand_dims(x):
    return K.expand_dims(x, 1)

def expand_dims_output_shape(input_shape):
    return (input_shape[0], 1, input_shape[1])

def LSTM_layer(input_):
    lstm1 = Lambda(expand_dims, expand_dims_output_shape)(input_)
    lstm1 = tf.keras.layers.ConvLSTM2D(filters=12, kernel_size=(3, 3), strides=(1, 1), data_format="channels_first",                             padding='same', return_sequences=False)(lstm1)
    return lstm1

具有功能性API的convLSTM2d

1 个答案: