Question

我正在尝试定义一个适用于图像序列的模型，并试图依次预测一个序列。我的问题是重复向量，特别是它的正确用法，其次是如何解决我得到的异常。

input_frames=Input(shape=(None, 128, 64, 1))
x=ConvLSTM2D(filters=64, kernel_size=(5, 5), padding='same', return_sequences=True)(input_frames)
x=BatchNormalization()(x)
x=ConvLSTM2D(filters=64, kernel_size=(5, 5), padding='same', return_sequences=False)(x)
x=BatchNormalization()(x)
x=Conv2D(filters=1, kernel_size=(5, 5), activation='sigmoid',padding='same')(x)
x=RepeatVector(10)(x)
model=Model(inputs=input_frames,outputs=x)

具体来说，我正在尝试预测未来的10帧。上面的代码引发以下异常：

assert_input_compatibility
str(K.ndim(x)))
ValueError: Input 0 is incompatible with layer repeat_vector_5: expected ndim=2, found ndim=4

Answer 1

在the doc of RepeatVector中，它仅接受2D输入，这就是错误消息告诉您的内容。

以下是使用Lambda层的解决方法：

from keras.layers import Input, ConvLSTM2D, BatchNormalization, RepeatVector, Conv2D
from keras.models import Model
from keras.backend import expand_dims, repeat_elements
from keras.layers import Lambda

input_frames=Input(shape=(None, 128, 64, 1))
x=ConvLSTM2D(filters=64, kernel_size=(5, 5), padding='same', return_sequences=True)(input_frames)
x=BatchNormalization()(x)
x=ConvLSTM2D(filters=64, kernel_size=(5, 5), padding='same', return_sequences=False)(x)
x=BatchNormalization()(x)
x=Conv2D(filters=1, kernel_size=(5, 5), activation='sigmoid',padding='same')(x)
#x=RepeatVector(10)(x)
x=Lambda(lambda x: repeat_elements(expand_dims(x, axis=1), 10, 1))(x)
model=Model(inputs=input_frames,outputs=x)
model.summary()

"""
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
input_15 (InputLayer)        (None, None, 128, 64, 1)  0         
_________________________________________________________________
conv_lst_m2d_5 (ConvLSTM2D)  (None, None, 128, 64, 64) 416256    
_________________________________________________________________
batch_normalization_5 (Batch (None, None, 128, 64, 64) 256       
_________________________________________________________________
conv_lst_m2d_6 (ConvLSTM2D)  (None, 128, 64, 64)       819456    
_________________________________________________________________
batch_normalization_6 (Batch (None, 128, 64, 64)       256       
_________________________________________________________________
conv2d_3 (Conv2D)            (None, 128, 64, 1)        1601      
_________________________________________________________________
lambda_1 (Lambda)            (None, 10, 128, 64, 1)    0         
=================================================================
Total params: 1,237,825
Trainable params: 1,237,569
Non-trainable params: 256
_________________________________________________________________
"""

请注意，我在这里使用expand_dims，因为从the doc of repeat_elements开始，它不会创建新轴。

如何使用重复向量预测输出序列？

1 个答案: