解决方案

Question

我想将2D稀疏矩阵转换为3D矩阵，因为我需要给它提供conv1d层作为输入，这需要3D张量。

这是conv1d层的输入。

from scipy.sparse import hstack
other_features_train = hstack((X_train_state_ohe, X_train_teacher_ohe, X_train_grade_ohe, X_train_category_ohe, X_train_subcategory_ohe,X_train_price_norm,X_train_number_norm))
other_features_cv = hstack((X_cv_state_ohe, X_cv_teacher_ohe, X_cv_grade_ohe,X_cv_category_ohe,X_cv_subcategory_ohe,X_cv_price_norm,X_cv_number_norm))
other_features_test = hstack((X_test_state_ohe, X_test_teacher_ohe, X_test_grade_ohe,X_test_category_ohe,X_test_subcategory_ohe,X_test_price_norm,X_test_number_norm))

print(other_features_train.shape)
print(other_features_cv.shape)
print(other_features_test.shape)

火车的形状，简历和测试数据

(49041, 101)
(24155, 101)
(36052, 101)

这是我的模型架构。

tf.keras.backend.clear_session()

vec_size = 300

input_model_1 = Input(shape=(300,),name='essay')
embedding = Embedding(vocab_size_essay, vec_size, weights=[word_vector_matrix], input_length = max_length, trainable=False)(input_model_1)
lstm = LSTM(16)(embedding)
flatten_1 = Flatten()(lstm)

input_model_2 = Input(shape=(101, ),name='other_features')
conv_layer1 = Conv1D(32, 3, strides=1, padding='valid', kernel_initializer='glorot_uniform', activation='relu')(input_model_2)
conv_layer2 = Conv1D(32, 3, strides=1, padding='valid', kernel_initializer='glorot_uniform', activation='relu')(conv_layer1)
conv_layer3 = Conv1D(32, 3, strides=1, padding='valid', kernel_initializer='glorot_uniform', activation='relu')(conv_layer2)
flatten_2 = Flatten()(conv_layer3)

concat_layer = concatenate(inputs=[flatten_1, flatten_2],name='concat')

dense_layer_1 = Dense(units=32, activation='relu', kernel_initializer='he_normal', name='dense_layer_1')(concat_layer)

dropout_1 = Dropout(0.2)(dense_layer_1)

dense_layer_2 = Dense(units=32, activation='relu', kernel_initializer='he_normal', name='dense_layer_2')(dropout_1)

dropout_2 = Dropout(0.2)(dense_layer_2)

dense_layer_3 = Dense(units=32, activation='relu', kernel_initializer='he_normal', name='dense_layer_3')(dropout_2)

output = Dense(units=2, activation='softmax', kernel_initializer='glorot_uniform', name='output')(dense_layer_3)

model_3 = Model(inputs=[input_model_1,input_model_2],outputs=output)

尝试提供2d数组时出现此错误。

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-18-44c8f6f0caa7> in <module>
      9 
     10 input_model_2 = Input(shape=(101, ),name='other_features')
---> 11 conv_layer1 = Conv1D(32, 3, strides=1, padding='valid', kernel_initializer='glorot_uniform', activation='relu')(input_model_2)
     12 conv_layer2 = Conv1D(32, 3, strides=1, padding='valid', kernel_initializer='glorot_uniform', activation='relu')(conv_layer1)
     13 conv_layer3 = Conv1D(32, 3, strides=1, padding='valid', kernel_initializer='glorot_uniform', activation='relu')(conv_layer2)

~\AppData\Local\Programs\Python\Python37\lib\site-packages\tensorflow_core\python\keras\engine\base_layer.py in __call__(self, inputs, *args, **kwargs)
    810         # are casted, not before.
    811         input_spec.assert_input_compatibility(self.input_spec, inputs,
--> 812                                               self.name)
    813         graph = backend.get_graph()
    814         with graph.as_default(), backend.name_scope(self._name_scope()):

~\AppData\Local\Programs\Python\Python37\lib\site-packages\tensorflow_core\python\keras\engine\input_spec.py in assert_input_compatibility(input_spec, inputs, layer_name)
    175                          'expected ndim=' + str(spec.ndim) + ', found ndim=' +
    176                          str(ndim) + '. Full shape received: ' +
--> 177                          str(x.shape.as_list()))
    178     if spec.max_ndim is not None:
    179       ndim = x.shape.ndims

ValueError: Input 0 of layer conv1d is incompatible with the layer: expected ndim=3, found ndim=2. Full shape received: [None, 101]

model_3.summary()
model_3.compile(loss = "binary_crossentropy", optimizer=Adam()

编译模型

model_3.compile(loss = "binary_crossentropy", optimizer=Adam(), metrics=["accuracy"])

适合模型

model_3.fit(train_features,y_train_ohe,batch_size=16,epochs=10,validation_data=(cv_features,y_cv_ohe))

train_features = [train_text, other_features_train]
cv_features = [cv_text, other_features_cv]
test_featues = [test_text, other_features_test]

文字功能

train_text = X_train['essay'].tolist()
cv_text = X_cv['essay'].tolist()
test_text = X_test['essay'].tolist()

token = Tokenizer()
token.fit_on_texts(train_text)

vocab_size_essay = len(token.word_index) + 1
print("No. of unique words = ", vocab_size_essay)

encoded_train_text = token.texts_to_sequences(train_text)
encoded_cv_text = token.texts_to_sequences(cv_text)
encoded_test_text = token.texts_to_sequences(test_text)

#print(encoded_test_text[:5])

max_length = 300

train_text = pad_sequences(encoded_train_text, maxlen=max_length, padding='post')
cv_text = pad_sequences(encoded_cv_text, maxlen=max_length, padding='post')
test_text = pad_sequences(encoded_test_text, maxlen=max_length, padding='post')

print("\n")
print(train_text.shape)
print(cv_text.shape)
print(test_text.shape)

文字特征的形状

No. of unique words =  41468


(49041, 300)
(24155, 300)
(36052, 300)

所以，我想在

中重塑

(49041,101,1) 
(24155,101,1) 
(36052,101,1)

请提出建议。

Answer 1

解决方案

这里的解决方案要求以下几个概念清晰。我将在以下各节中解释这些概念。

keras期望作为输入
可以对您的keras模型进行什么样的修改以允许稀疏输入矩阵
将2D numpy数组转换为3D numpy数组
来回转换
- scipy.sparse.coo_matrix用于2D numpy阵列
- sparse.COO用于3D numpy阵列

使用稀疏矩阵作为`tf.keras`模型的输入

一种选择是使用todense()方法将稀疏输入矩阵转换为非稀疏（密集）格式。这使矩阵成为常规的numpy数组。参见kaggle讨论[3]和[4]。
另一种选择是通过子类tf.keras.layers.Layer的类为稀疏和密集输入编写自己的自定义图层。参见这篇文章[2]。
看来tensorflow.keras现在允许使用稀疏权重进行模型训练。因此，它具有处理稀疏性的能力。您可能需要浏览此方面的文档[{1]。

向numpy数组添加新轴

您可以使用np.newaxis将另一个轴添加到numpy数组，如下所示。

import numpy as np

## Make a 2D array
a2D = np.zeros((10,10))

# Make a few elements non-zero in a2D
aa = a2D.flatten()
aa[[0,13,41,87,98]] = np.random.randint(1,10,size=5)
a2D = aa.reshape(a2D.shape)

# Make 3D array from 2D array by adding another axis
a3D = a2D[:,:,np.newaxis]
#print(a2D)
print('a2D.shape: {}\na3D.shape: {}'.format(a2D.shape, a3D.shape))

输出：

a2D.shape: (10, 10)
a3D.shape: (10, 10, 1)

话虽如此，请查看参考部分中的链接。

稀疏数组

由于稀疏数组具有很少的非零值，因此将常规的numpy数组转换为稀疏数组后，会将其存储为几种稀疏格式：

csr_matrix：非零值和索引的按行排列
csc-matrix：非零值和索引的列式数组
coo-matrix：具有三列的表格
- 行
- 列
- 非零值

稀疏矩阵期望2D输入矩阵

但是，上述三种类型的稀疏矩阵的scipy.sparse实现仅将2D非稀疏矩阵作为输入。

from scipy.sparse import csr_matrix, coo_matrix

coo_a2D = coo_matrix(a2D)
coo_a2D.shape # output: (10, 10)

# scipy.sparse only accepts 2D input matrices
# the following line will throw an !!! ERROR !!!
coo_a3D = coo_matrix(coo_a2D.todense()[:,:,np.newaxis])

3D非稀疏输入矩阵中的稀疏矩阵

是的，您可以使用sparse库进行此操作。它还支持scipy.sparse和numpy数组。要将稀疏矩阵转换为非稀疏（密集）格式（这不是神经网络中的密集层），请使用todense()方法。

## Installation
# pip install -U sparse

import sparse

## Create sparse coo_matrix from a
# 3D numpy array (dense format)
coo_a3D = sparse.COO(a3D)

## Test that
#   coo_a3D == coo made from (coo_a2D + newaxis)
print(
    (coo_a3D == sparse.COO(coo_a2D.todense()[:,:,np.newaxis])).all()
) # output: True
## Convert to dense (non-sparse) format
#   use: coo_a3D.todense()
print((a3D == coo_a3D.todense()).all()) # output: True

Source

参考文献

Answer 2

您可以简单地使用np.reshape

https://numpy.org/doc/1.18/reference/generated/numpy.reshape.html

other_features_train = other_features_train.reshape(other_features_train.shape[0], other_features_train.shape[1], 1)

other_features_cv = other_features_cv.reshape(other_features_cv.shape[0], other_features_cv.shape[1], 1)

other_features_test = other_features_test.reshape(other_features_test.shape[0], other_features_test.shape[1], 1)

此外，您需要更改此行

input_model_2 = Input(shape=(101, 1),name='other_features')

Conv1D需要3d数据，而不是2d。

将2D稀疏矩阵转换为3D矩阵

2 个答案:

解决方案

使用稀疏矩阵作为tf.keras模型的输入

向numpy数组添加新轴

稀疏数组

3D非稀疏输入矩阵中的稀疏矩阵

参考文献

使用稀疏矩阵作为`tf.keras`模型的输入