Question

This is my first time creating a multi-input keras model, and I am not sure if I am doing something wrong as my training accuracy is not rising above 50 percent. If I use a single-input model, I can get to 90 percent plus pretty easily. This model uses the same input but different sizes of the input and then takes the max of the softmax prediction as the final prediction. So, I would assume it would be easier to train the model.

Here is the code:

def full_model_2(model_key):
    base_model, preprocess = basemodel(model_key)  #base model is just an imported imagenet model
    base_model2, preprocess = basemodel(model_key)
for layer in base_model.layers:
    layer.name = layer.name + "dup"

#first model    
x = base_model.output
x = GlobalAveragePooling2D()(x)
x = Dense(2048, activation='relu')(x)
x = BatchNormalization()(x)
x =  Dense(512, activation='relu')(x)
x = BatchNormalization()(x)
predictions_x = Dense(classes, activation='softmax')(x)

#second model
y = base_model2.output
y = GlobalAveragePooling2D()(y)
y = Dense(2048, activation='relu')(y)
y = BatchNormalization()(y)
y =  Dense(512, activation='relu')(y)
y = BatchNormalization()(y)
predictions_y = Dense(classes, activation='softmax')(y)
#predictions = Average()([predictions_x, predictions_y])
predictions = Maximum()([predictions_x, predictions_y])
model = Model(inputs= [base_model.input,base_model2.input], outputs=predictions)
for layer in model.layers[:-10]:
    layer.trainable = False
return model, preprocess

My generator looks something like this (take two generators and creates two inputs)

def train_generator():
    train_gen, _ = create_generators(batch_size, preprocess, img_size)
    train_gen2, _ = create_generators(batch_size, preprocess, img_size2)
    print(train_gen.seed)
    print(train_gen2.seed)
    while True:
        X1i = train_gen.next()
        X2i = train_gen2.next()
        yield [X1i[0], X2i[0]], X1i[1]

Here is the original generator function:

train_datagen = ImageDataGenerator(preprocessing_function = preprocess)
    train_generator = train_datagen.flow_from_directory(
    train_data_dir,
    shuffle = True,
    seed = seed,
    target_size = (img_size, img_size),
    batch_size = batch_size, 
    class_mode = 'categorical')

return train_generator


 model.compile(optimizer= r_optimizer,
              loss='categorical_crossentropy',
              metrics = ['accuracy'])

I have plotted the inputs from the generators, and I am indeed getting the same images but different sizes, so the input from generators seems to be okay.

I am thinking my model setup is off.

Thank you for your suggestions.

Multi-input Keras model resulting in low accuracy (is my model setup incorrect?)

0 个答案: