提供ValueError的多输入CNN的keras自定义生成器：形状（None，146，73，2）和（None，None）不兼容

Question

我正在尝试创建具有多个输入的CNN模型，如下所示。

p_input = Input(shape=(600,600,1))
p_branch = Conv2D(32, (3, 3), activation='relu')(p_input)
p_branch = MaxPooling2D(pool_size=(2, 2))(p_branch)
p_branch = Conv2D(32, (3, 3), activation='relu')(p_branch)
p_branch = MaxPooling2D(pool_size=(2, 2))(p_branch)
p_branch = Conv2D(32, (3, 3), activation='relu')(p_branch)
p_branch = MaxPooling2D(pool_size=(2, 2))(p_branch)
#p_branch = Flatten()(p_branch)

s_input = Input(shape=(600,600,1))
s_branch = Conv2D(32, (3, 3), activation='relu')(s_input)
s_branch = MaxPooling2D(pool_size=(2, 2))(s_branch)
s_branch = Conv2D(32, (3, 3), activation='relu')(s_branch)
s_branch = MaxPooling2D(pool_size=(2, 2))(s_branch)
s_branch = Conv2D(32, (3, 3), activation='relu')(s_branch)
s_branch = MaxPooling2D(pool_size=(2, 2))(s_branch)
#s_branch = Flatten()(s_branch)

concat = Concatenate(axis=1)([p_branch, s_branch])

fusion = Flatten()(concat)
fusion = Dense(64, activation='relu')(concat)
fusion = Dropout((0.5))(fusion)
fusion = Dense(32, activation='relu')(fusion)
fusion = Dropout((0.5))(fusion)
output = Dense(2, activation='sigmoid')(fusion)


model = Model(inputs=[p_input, s_input], outputs =[output])

model.compile(loss='binary_crossentropy', optimizer='rmsprop', metrics=METRICS)

为此，我创建了一个自定义生成器，如下所示：-

original_train = "image_path"
data_path = "csv_file_path"

def get_flow_from_dataframe(generator,dataframe,
                            image_shape=(600, 600),
                            subset='training',
                            batch_size=32, color_mode='grayscale'):
    train_generator_1 = generator.flow_from_dataframe(dataframe, target_size=image_shape,
                                                      color_mode='grayscale',
                                                      directory=original_train,
                                                      x_col='Id2',
                                                      y_col='label2',
                                                      class_mode='categorical',
                                                      shuffle=False,
                                                      batch_size=batch_size,
                                                      seed=1,
                                                      subset=subset)

    train_generator_2 = generator.flow_from_dataframe(dataframe, target_size=image_shape,
                                                      directory=original_train,
                                                      color_mode='grayscale',
                                                      x_col='Id1',
                                                      y_col='label1',
                                                      class_mode='categorical',
                                                      shuffle=False,
                                                      batch_size=batch_size,
                                                      seed=1,
                                                      subset=subset)
    
    while True:
        x_1 = train_generator_1.next()
        x_2 = train_generator_2.next()
        yield [x_1[0],x_2[0]], x_1[1]

我有一个包含图像名称和标签的数据框，例如：

|---------------------|------------------|------------------|------------------|
|          ID1        |      label1      |       ID2        |      label2      |
|---------------------|------------------|------------------|------------------|
|        img1.png     |      class1      |     img2.png     |      class2      |
|---------------------|------------------|------------------|------------------|
|        img2.png     |      class2      |     img3.png     |      class2      |
|---------------------|------------------|------------------|------------------|
|        img3.png     |      class2      |     img4.png     |      class1      |
|---------------------|------------------|------------------|------------------|

因此，Id1是图像序列中Id2的前一个图像。

然后我创建了生成器：-

data = pd.read_csv(data_path)

train_set, test_set = train_test_split(data, test_size=0.2, random_state=0, shuffle=False)

generator = ImageDataGenerator(rescale=1. / 255, validation_split=0.2)
train_gen = get_flow_from_dataframe(generator,data, image_shape=(600,600), batch_size=batch_size)
valid_gen = get_flow_from_dataframe(generator,data, image_shape=(600,600), batch_size=batch_size, subset='validation')

train_steps = train_set.shape[0]//batch_size
validation_steps = test_set.shape[0]//batch_size

history = model.fit(train_gen,steps_per_epoch=train_steps, epochs=epochs,
                              validation_data=valid_gen,validation_steps=validation_steps)

但是以某种方式我会收到此错误：-

ValueError: Shapes (None, 146, 73, 2) and (None, None) are incompatible

或

ValueError: logits and labels must have the same shape ((None, 146, 73, 2) vs (None, None))