运行sklearn分类器模型时的值错误

时间:2017-06-20 06:33:44

标签: python scikit-learn nlp

我是sklearn的新手,我正在尝试使用scikit构建一个简单的文本分类器,但遇到了ValueError。它在fit()显示错误,但其他教程正在使用它并且运行正常。

这是我的代码:

from sklearn.datasets import fetch_20newsgroups
from sklearn.cross_validation import train_test_split
from sklearn.feature_extraction.text import TfidfVectorizer
from sklearn.pipeline import Pipeline
from sklearn.naive_bayes import MultinomialNB

news = fetch_20newsgroups(subset='all')
print len(news.data)



def train(classifier , X , y):
        X_train , y_train , X_test , y_test = train_test_split(X,y,test_size =            0.20, random_state = 33)
        classifier.fit(X_train ,y_train)
        print "Accuracy %s" % classifier.score(X_test , y_test)
        return classifier

model1 = Pipeline([('vectorizer' , TfidfVectorizer()),('classifier' , MultinomialNB()),])

train(model1 , news.data , news.target)

运行时,我收到了值错误

Traceback (most recent call last):
  File "/home/padam/Documents/git/ticketClassifier/news.py", line 30, in <module>
    train(model1 , news.data , news.target)
  File "/home/padam/Documents/git/ticketClassifier/news.py", line 24, in train
    classifier.fit(X_train ,y_train)
  File "/usr/lib/python2.7/dist-packages/sklearn/pipeline.py", line 165, in fit
    self.steps[-1][-1].fit(Xt, y, **fit_params)
  File "/usr/lib/python2.7/dist-packages/sklearn/naive_bayes.py", line 527, in fit
    X, y = check_X_y(X, y, 'csr')
  File "/usr/lib/python2.7/dist-packages/sklearn/utils/validation.py", line 520, in check_X_y
    check_consistent_length(X, y)
  File "/usr/lib/python2.7/dist-packages/sklearn/utils/validation.py", line 176, in check_consistent_length
    "%s" % str(uniques))
ValueError: Found arrays with inconsistent numbers of samples: [ 3770 15076]

样本数量不一致是什么意思。其他stackoverflow解决方案建议重新排列numpy矩阵的矩阵。但我没有使用numpy。 谢谢!

1 个答案:

答案 0 :(得分:2)

错误在于您使用train_test_split

的方式

您正在使用它

X_train , y_train , X_test , y_test = train_test_split(X, y,
                                                 test_size = 0.20, 
                                                 random_state = 33)

但实际输出顺序as given in documentation不同。它是:

X_train , X_test , y_train ,  y_test = train_test_split(X, y,
                                                 test_size = 0.20, 
                                                 random_state = 33)

另外,建议如果您使用的是scikit版本&gt; = 0.18,请将软件包从cross_validation更改为model_selection,因为它已弃用,将在新版本中删除。

所以而不是: -

from sklearn.cross_validation import train_test_split

使用以下内容:

from sklearn.model_selection import train_test_split