与Keras合二为一

Question

在Keras上是否可以将图像和值向量组合在一起？如果是，怎么办？

我想要的是创建一个CNN，其中包含一个图像和一个在输入中包含6个值的向量。

输出为3个值。

Answer 1

是的，请查看Keras的Functional API，以获取有关如何构建具有多个输入的模型的许多示例。

您的代码看起来像这样，您可能希望将图像传递给卷积层，将输出展平并与向量输入连接起来：

from keras.layers import Input, Concatenate, Conv2D, Flatten, Dense
from keras.models import Model

# Define two input layers
image_input = Input((32, 32, 3))
vector_input = Input((6,))

# Convolution + Flatten for the image
conv_layer = Conv2D(32, (3,3))(image_input)
flat_layer = Flatten()(conv_layer)

# Concatenate the convolutional features and the vector input
concat_layer= Concatenate()([vector_input, flat_layer])
output = Dense(3)(concat_layer)

# define a model with a list of two inputs
model = Model(inputs=[image_input, vector_input], outputs=output)

这将为您提供一个具有以下规格的模型：

Layer (type)                    Output Shape         Param #     Connected to                     
==================================================================================================
input_8 (InputLayer)            (None, 32, 32, 3)    0                                            
__________________________________________________________________________________________________
conv2d_4 (Conv2D)               (None, 30, 30, 32)   896         input_8[0][0]                    
__________________________________________________________________________________________________
input_9 (InputLayer)            (None, 6)            0                                            
__________________________________________________________________________________________________
flatten_3 (Flatten)             (None, 28800)        0           conv2d_4[0][0]                   
__________________________________________________________________________________________________
concatenate_3 (Concatenate)     (None, 28806)        0           input_9[0][0]                    
                                                                 flatten_3[0][0]                  
__________________________________________________________________________________________________
dense_3 (Dense)                 (None, 3)            86421       concatenate_3[0][0]              
==================================================================================================
Total params: 87,317
Trainable params: 87,317
Non-trainable params: 0

另一种可视化方法是通过Keras' visualization utilities：