Question

我是NLP和深度学习的新手，所以我有一个（很可能是）非常基本的问题。

我正在尝试基于预训练的BERT嵌入作为特征来创建二进制分类器。到目前为止，我已经成功创建了嵌入并使用tensorflow.keras构建了一个简单的Sequential（）模型。下面的代码有效：

model = tf.keras.Sequential([
    Dense(4, activation = 'relu', input_shape = (768,)),
    Dense(4, activation = 'relu'),
    Dense(1, activation = 'sigmoid')])

model.compile(optimizer = 'adam',
              loss = 'binary_crossentropy',
              metrics = ['accuracy'])

我想做的就是将这段代码改编为现在的CNN。但是，当我添加卷积层时，会出现错误：

model = tf.keras.Sequential([
    Conv1D(filters = 250, kernel_size = 3, padding='valid', activation='relu', strides=1, input_shape = (768,)),
    GlobalMaxPooling1D(),
    Dense(4, activation = 'relu'),
    Dense(1, activation = 'sigmoid')])

model.compile(optimizer = 'adam',
              loss = 'binary_crossentropy',
              metrics = ['accuracy'])

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-23-59695050a94e> in <module>()
      3     GlobalMaxPooling1D(),
      4     Dense(4, activation = 'relu'),
----> 5     Dense(1, activation = 'sigmoid')])
      6 
      7 model.compile(optimizer = 'adam',

5 frames
/usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/engine/input_spec.py in assert_input_compatibility(input_spec, inputs, layer_name)
    178                          'expected ndim=' + str(spec.ndim) + ', found ndim=' +
    179                          str(ndim) + '. Full shape received: ' +
--> 180                          str(x.shape.as_list()))
    181     if spec.max_ndim is not None:
    182       ndim = x.shape.ndims

ValueError: Input 0 of layer conv1d_2 is incompatible with the layer: expected ndim=3, found ndim=2. Full shape received: [None, 768]

这是我正在使用的数据的样子。

功能：

train_features[0]

array([-4.97862399e-01,  1.49541467e-01,  5.81708886e-02,  1.63668215e-01,
       -2.77605206e-01,  3.57868642e-01,  1.70950562e-01,  2.69330859e-01,
       -3.29369396e-01,  2.12891083e-02, -4.02462274e-01, -1.98120754e-02,
       -2.18944401e-01,  4.34780568e-01, -2.75409579e-01,  2.03015730e-01,...

train_features[0].shape
(768,)

标签：

train_labels.iloc[0:3]
turnout       
0        73446    0
1        53640    1
         16895    1
Name: turnout, dtype: int64

任何建议都非常感谢。非常感谢！

Answer 1

2D卷积需要4D输入：(batch_size, width1, width2, channels)。

您的数据是一个形状为(batch_size, 768)的单个数组。如果您确实想使用卷积（如果您认为数据中可能存在空间关系），则需要在将其输入模型之前对其进行适当的成形。

一维卷积需要3D输入：(batch_size, length, channels)。

使用经过预训练的BERT嵌入作为带有tensorflow.keras的CNN输入会导致ValueError

1 个答案: