首先,我将TfidfVectorization应用于数据框中的文本列。之后,我使用hsatck合并它们
train_gene_var_tfidf =
hstack((train_gene_feature_tfidf,train_variation_feature_tfidf))
test_gene_var_tfidf =
hstack((test_gene_feature_tfidf,test_variation_feature_tfidf))
cv_gene_var_tfidf =
hstack((cv_gene_feature_tfidf,cv_variation_feature_tfidf))
# Adding the train_text feature
train_x_tfidf = hstack((train_gene_var_tfidf, train_text))
train_x_tfidf = hstack((train_x_tfidf, train_text_feature_tfidf))
train_y = np.array(list(train_df['Class']))
结果是我得到了一个包含稀疏矩阵的numpy ndarray 我想让我的函数适合sklearn .fit(),如下所示
clf = SGDClassifier(class_weight='balanced',alpha=i,penalty='l2', loss='log', random_state=42)
clf.fit(train_x_tfidf, train_y)
形状如下:
print(train_x_tfidf.shape)
print(train_y.shape)
>>>(4,)
>>>(2124,)
print(train_x_tfidf)
>>>[<2124x227 sparse matrix of type '<class 'numpy.float64'>'
with 2124 stored elements in Compressed Sparse Row format>
<2124x1000 sparse matrix of type '<class 'numpy.float64'>'
with 1259 stored elements in Compressed Sparse Row format>
<2124x1000 sparse matrix of type '<class 'numpy.float64'>'
with 31088 stored elements in Compressed Sparse Row format>
<2124x1000 sparse matrix of type '<class 'numpy.float64'>'
with 1129081 stored elements in Compressed Sparse Column format>]
我在适应过程中遇到的错误是
ValueError:设置具有序列的数组元素。
如何将此ndarray放入sklearn fit函数中?