组合多个张量流模型

Question

我是深度学习和张量流的新手。而且似乎所有在线示例都是非常简单的顺序模型。但是我有一个有点复杂的模型，我正在尝试使用tensorflow 2.1实现。因此，简而言之，我有两个模型A，B，我正尝试将其输出合并并用作模型C的输入。请参考所附的图表，以更清楚地了解我要实现的模型架构{{3 }} ve

到目前为止，这里还有未完成的代码，我仍然遇到很多错误。关于如何在网络下实施的任何建议。预先感谢。

def model_a():
    model_small = Sequential()
    model_small.add(Conv1D(filters=64, kernel_size=50, activation=None, input_shape=(3000, 1)))
    model_small.add(MaxPool1D(pool_size=8, strides=8))
    model_small.add(Dropout(0.5))
    model_small.add(Conv1D(filters=128, kernel_size=8, strides=1, activation=None))
    model_small.add(Conv1D(filters=128, kernel_size=8, strides=1, activation=None))
    model_small.add(Conv1D(filters=128, kernel_size=8, strides=1, activation=None))
    model_small.add(MaxPool1D(pool_size=4, strides=4))
    model_small.add(Flatten())
    return model_small
    #return model_small.add(Flatten())

def model_c():
    model = Sequential()
    model.add(Bidirectional(LSTM(512)))
    model.add(Dropout(0.5))
    model.add(Dense(4, activation='sigmoid'))

def model_b():
    model_large = Sequential()
    # the number of strides in this layer is too large at 50?
    model_large.add(Conv1D(filters=64, kernel_size=400, activation=None, input_shape=(3000, 1)))
    model_large.add(MaxPool1D(pool_size=4))
    model_large.add(Dropout(0.5))
    model_large.add(Conv1D(filters=128, kernel_size=6, activation=None))
    model_large.add(Conv1D(filters=128, kernel_size=6, activation=None))
    model_large.add(Conv1D(filters=128, kernel_size=6, strides=1, activation=None))
    model_large.add(MaxPool1D(pool_size=2))
    model_large.add(Flatten())
    return model_large
    #return model_large.add(Flatten())

def final_model():
    input1 = model_a()
    input2 = model_b()
    model_concat = concatenate([input1.output, input2.output], axis=1)
    model_concat = Dropout(0.5)(model_concat)
    # try to fix model_c here but i don't how 
    model= Model(inputs=[input1.input, input2.input], outputs=model_concat)

    return model

model_2 = final_model()
model_2.compile(
    loss=tf.keras.losses.sparse_categorical_crossentropy,
    optimizer=tf.keras.optimizers.SGD(learning_rate=0.01),
    metrics=['accuracy']  # can add more metrics
)


model_2.fit(x=INPUT, epochs=10)

Answer 1

您在正确的轨道上。您可以通过两种方式完成：建立一个分为功能部分的模型，或者建立多个模型并将它们链接在一起。从功能上讲，它们是同一件事，因为层是模型。模型可以是一个层。

我将使用一些简单的Dense层来表示model_a到model_c。请注意，如果使用顺序api，您可能会遇到这样的错误，因此我将以功能性api进行演示，但是我向您保证，它同样易于使用（该模型仅在定义之后层，而不是之前）。

由于一个模型分为多个功能：

import tensorflow as tf

def model_a(x):
  return tf.keras.layers.Dense(56,name='model_a_whatever')(x) # returning output of layer

def model_b(x):
  x = tf.keras.layers.Dense(56,name='model_b_whatever')(x)
  return tf.keras.layers.Dense(56,name='model_b_more_whatever')(x)

def model_c(x):
  x = tf.keras.layers.Dense(56,name='model_c_whatever')(x)
  x = tf.keras.layers.Dense(56,name='model_c_more_whatever')(x)
  return tf.keras.layers.Dense(56,name='model_c_even_more_whatever')(x)

# in through the input layer
main_input = tf.keras.layers.Input(shape=(12,34,56),name='INPUT')

# now through functions containing different models' layers
left = model_a(main_input)
right = model_b(main_input)

# concatenate their outputs
concatenated = tf.keras.layers.Concatenate(axis=-1)([left,right])

# now through function containing layers of model c
left = model_c(concatenated)

# and the juke right to a fully connected layer
right = tf.keras.layers.Dense(56,name='FC')(concatenated)

# then add the outputs and apply softmax activation
added = tf.keras.layers.Add(name='add')([left,right])
outputs = tf.keras.layers.Activation('softmax',name='Softmax')(added)

# now define the model
model = tf.keras.models.Model(main_input,outputs) # Model(input layer, final output))

print(model.summary())
tf.keras.utils.plot_model(model, to_file='just_a_model.png')

该图将比您的图更加混乱，因为所有层都可见：

许多模型都加入到一起：

# as separate models linked together

def build_model_a():
    a = tf.keras.layers.Input(shape=(12,34,56),name='model_a_input')
    b = tf.keras.layers.Dense(56,name='model_a_whatever')(a) # whatever layers
    return tf.keras.models.Model(a,b,name='MODEL_A') # returning model, not just layer output

def build_model_b():
  a = tf.keras.layers.Input(shape=(12,34,56),name='model_b_input')
  b = tf.keras.layers.Dense(56,name='model_b_whatever')(a)
  b = tf.keras.layers.Dense(56,name='model_b_more_whatever')(b)
  return tf.keras.models.Model(a,b,name='MODEL_B')

def build_model_c():
  a = tf.keras.layers.Input(shape=(12,34,112),name='model_c_input') # axis 2 is doubled because concatenation.
  b = tf.keras.layers.Dense(56,name='model_c_whatever')(a)
  b = tf.keras.layers.Dense(56,name='model_c_more_whatever')(b)
  b = tf.keras.layers.Dense(56,name='model_c_even_more_whatever')(b)
  return tf.keras.models.Model(a,b,name='MODEL_C')

# define the main input
main_input = tf.keras.layers.Input(shape=(12,34,56),name='INPUT')

# build the models
model_a = build_model_a()
model_b = build_model_b()
model_c = build_model_c()

# pass input through models a and b
a = model_a(main_input)
b = model_b(main_input)

# concatenate their outputs
ab = tf.keras.layers.Concatenate(axis=-1,name='Concatenate')([a,b])

# pass through model c and fully-connected layer
c = model_c(ab)
d = tf.keras.layers.Dense(56,name='FC')(ab)

# add their outputs and apply softmax activation
add = tf.keras.layers.Add(name="add")([c,d])
outputs = tf.keras.layers.Activation('softmax',name='Softmax')(add)

model = tf.keras.models.Model(main_input,outputs)

print(model.summary())
tf.keras.utils.plot_model(model, to_file='multi_model.png')

尽管在功能上与第一种情况相同，但该图现在与您的图相匹配：

这两种方法都可以。正如您所看到的，第一种方法实际上只是代码清除。为了清楚起见，将单独的数据管道放入函数中。如果您想变得更复杂，例如对子模型具有不同的损失函数，那么第二种方法可能会简化过程。我提到的错误仅在将序贯api与第二种方法结合使用时发生。