Question

我正在研究Scikit-learn中的朴素贝叶斯分类器。

在训练期间和预测阶段我都使用以下代码从元组列表中获取csr_matrix：

def convert_to_csr_matrix(vectors):
    """
    convert list of tuples representation to scipy csr_matrix that is needed
    for scikit learner
    """
    logger.info("building the csr_sparse matrix representing tf-idf")
    row = [[i] * len(v) for i, v in enumerate(vectors)]
    row = list(chain(*row))
    column = [j for j, _ in chain(*vectors)]
    data = [d for _, d in chain(*vectors)]
    return csr_matrix((data, (row, column)))

我实施的主要基于scipy csr_matrix from several vectors represented as list of sets

不幸的是，现在在预测阶段，我收到以下错误：

File "/Users/zikes/project/taxonomy_data_preprocessing/single_classification.py", line 93, in predict
top_predictions = self.top.predict(item)
File "/Users/zikes/project/taxonomy_data_preprocessing/single_classification.py", line 124, in predict
category, res = model.predict(item)
File "/Users/zikes/project/taxonomy_data_preprocessing/single_classification.py", line 176, in predict
prediction = self.clf.predict(item)
File "/Users/zikes/.virtualenvs/taxonomy/lib/python2.7/site-packages/sklearn/naive_bayes.py", line 64, in predict
jll = self._joint_log_likelihood(X)
File "/Users/zikes/.virtualenvs/taxonomy/lib/python2.7/site-packages/sklearn/naive_bayes.py", line 615, in _joint_log_likelihood
return (safe_sparse_dot(X, self.feature_log_prob_.T)
File "/Users/zikes/.virtualenvs/taxonomy/lib/python2.7/site-packages/sklearn/utils/extmath.py", line 178, in safe_sparse_dot
ret = a * b
File "/Users/zikes/.virtualenvs/taxonomy/lib/python2.7/site-packages/scipy/sparse/base.py", line 354, in __mul__
raise ValueError('dimension mismatch')
ValueError: dimension mismatch

有谁知道什么是错的？我想，不知何故，稀疏的向量有错误的尺寸。但我不明白为什么？

在调试过程中，我在Naive Bayes模型的feature_log_prob_日志中打印出来，看起来像：

[[-11.82052115 -12.51735721 -12.51735721 ..., -12.51735721 -11.60489688
-12.2132116 ]
[-12.21403023 -12.51130295 -12.51130295 ..., -11.84156341 -12.51130295
-12.51130295]]

shape：(2, 53961)

我要预测csr_matrix = (0, 7637) 0.770238101052 (0, 21849) 0.637756432886

表示为元组列表，它看起来像：[(7637, 0.7702381010520318), (21849, 0.6377564328862234)]

Answer 1

因此，在对问题进行了一些调查之后，我意识到可能的解决方法可能在方法中：

def convert_to_csr_matrix(vectors):
   """
   convert list of tuples representation to scipy csr_matrix that is needed
   for scikit learner
   """
   logger.info("building the csr_sparse matrix representing tf-idf")
   row = [[i] * len(v) for i, v in enumerate(vectors)]
   row = list(chain(*row))
   column = [j for j, _ in chain(*vectors)]
   data = [d for _, d in chain(*vectors)]
   return csr_matrix((data, (row, column)))

行return csr_matrix((data, (row, column)))应替换为return csr_matrix((data, (row, column)), shape=(len(vectors), dimension))

Scikit学习Naive Bayes ValueError：维度不匹配

1 个答案: