我正在处理一个二进制分类问题，我在以后通过预先训练的嵌入传递数据，然后并行并行几个CNN，合并结果，然后使用密集层进行预测方面相当成功班级。但是，当我改为在CNN后面放置RNN时，训练完全失败了。下面的代码（这是一个很长的帖子）。
这是仅适用于CNN的模型。我的输入是长度为100的向量。
inputs=L.Input(shape=(100))
embedding=L.Embedding(input_dim=weights.shape[0],\
output_dim=weights.shape[1],\
input_length=100,\
weights=[weights],\
trainable=False)(inputs)
conv3 = L.Conv1D(m, kernel_size=(3))(dropout)
conv4 = L.Conv1D(m, kernel_size=(4))(dropout)
conv5 = L.Conv1D(m, kernel_size=(5))(dropout)
maxpool3 = L.MaxPool1D(pool_size=(100-3+1, ), strides=(1,))(conv3)
maxpool4 = L.MaxPool1D(pool_size=(100-4+1, ), strides=(1,))(conv4)
maxpool5 = L.MaxPool1D(pool_size=(100-5+1, ), strides=(1,))(conv5)
concatenated_tensor = L.Concatenate(axis=1)([maxpool3,maxpool4,maxpool5])
flattened = L.Flatten()(concatenated_tensor)
output = L.Dense(units=1, activation='sigmoid')(flattened)
这是摘要：
____________________________________________________________________________________________________
Layer (type) Output Shape Param # Connected to
====================================================================================================
input_25 (InputLayer) (None, 100) 0
____________________________________________________________________________________________________
embedding_25 (Embedding) (None, 100, 50) 451300 input_25[0][0]
____________________________________________________________________________________________________
dropout_25 (Dropout) (None, 100, 50) 0 embedding_25[0][0]
____________________________________________________________________________________________________
conv1d_73 (Conv1D) (None, 98, 100) 15100 dropout_25[0][0]
____________________________________________________________________________________________________
conv1d_74 (Conv1D) (None, 97, 100) 20100 dropout_25[0][0]
____________________________________________________________________________________________________
conv1d_75 (Conv1D) (None, 96, 100) 25100 dropout_25[0][0]
____________________________________________________________________________________________________
max_pooling1d_73 (MaxPooling1D) (None, 1, 100) 0 conv1d_73[0][0]
____________________________________________________________________________________________________
max_pooling1d_74 (MaxPooling1D) (None, 1, 100) 0 conv1d_74[0][0]
____________________________________________________________________________________________________
max_pooling1d_75 (MaxPooling1D) (None, 1, 100) 0 conv1d_75[0][0]
____________________________________________________________________________________________________
concatenate_25 (Concatenate) (None, 3, 100) 0 max_pooling1d_73[0][0]
max_pooling1d_74[0][0]
max_pooling1d_75[0][0]
____________________________________________________________________________________________________
flatten_25 (Flatten) (None, 300) 0 concatenate_25[0][0]
____________________________________________________________________________________________________
dense_47 (Dense) (None, 1) 301 flatten_25[0][0]
====================================================================================================
正如我在上面说的那样，此方法效果很好，仅在3-4个纪元后就获得了良好的准确性。但是，我的思维过程是CNN可以识别区域模式，但是如果我还想对给定输入向量中的模式在更长的距离上如何相互关联进行建模，那么在卷积之后我应该使用某种RNN的风格。因此，我尝试在卷积后更改pool_size层中的MaxPooling1D，删除了Flatten，而是将Concatenate层传递到RNN中。例如
maxpool3 = L.MaxPool1D(pool_size=((50,), strides=(1,))(conv3)
maxpool4 = L.MaxPool1D(pool_size=((50,), strides=(1,))(conv4)
maxpool5 = L.MaxPool1D(pool_size=(49,), strides=(1,))(conv5)
concatenated_tensor = L.Concatenate(axis=1)([maxpool3,maxpool4,maxpool5])
rnn=L.SimpleRNN(75)(concatenated_tensor)
output = L.Dense(units=1, activation='sigmoid')(rnn)
摘要现在变为：
max_pooling1d_95 (MaxPooling1D) (None, 50, 100) 0 conv1d_97[0][0]
____________________________________________________________________________________________________
max_pooling1d_96 (MaxPooling1D) (None, 50, 100) 0 conv1d_98[0][0]
____________________________________________________________________________________________________
max_pooling1d_97 (MaxPooling1D) (None, 49, 100) 0 conv1d_99[0][0]
____________________________________________________________________________________________________
concatenate_32 (Concatenate) (None, 149, 100) 0 max_pooling1d_95[0][0]
max_pooling1d_96[0][0]
max_pooling1d_97[0][0]
____________________________________________________________________________________________________
simple_rnn_5 (SimpleRNN) (None, 75) 13200 concatenate_32[0][0]
____________________________________________________________________________________________________
dense_51 (Dense) (None, 1) 76 simple_rnn_5[0][0]
====================================================================================================
当我训练模型时，所有预测都完全相同：class [1]与class [0]之比。我读过几篇论文，人们成功地使用了该方案，因此很明显我做错了，我敢打赌这是一个令人尴尬的愚蠢错误。有人愿意帮助诊断吗？
答案 0 :(得分：0)
您可以尝试的第一件事是沿着特征轴而不是时间轴进行连接。 基本上试试这个：
maxpool3 = L.MaxPool1D(pool_size=(50,), strides=(1,))(conv3)
maxpool4 = L.MaxPool1D(pool_size=(50,), strides=(1,))(conv4)
maxpool5 = L.MaxPool1D(pool_size=(50,), strides=(1,))(conv5)
concatenated_tensor = L.Concatenate(axis=2)([maxpool3,maxpool4,maxpool5])
rnn=L.SimpleRNN(75)(concatenated_tensor)
output = L.Dense(units=1, activation='sigmoid')(rnn)
（请注意，您必须确保maxpool3，maxpool4和maxpool5的“时间”步数相同或maxpool3.shape [1] = maxpool4.shape [1] = maxpool5.shape [1]）< / p>
其次，以50个时间步长，为LSTM或GRU提供了一个机会，因为它们可以比LSTM更好地捕获更长的时间依赖性。