我是scikit-learn的新手。我需要计算大型语料库的tf-idf向量。但在开始之前,我尝试编写一些小文档,每个文档不超过5-6个单词。我写的具体代码如下:
from sklearn.feature_extraction.text
import TfidfVectorizer
vectorizer = TfidfVectorizer(min_df=1)
vectors = vectorizer.fit_transform(docList)
我的笔记本电脑运行正常,但在服务器上运行时会产生以下错误:
Traceback (most recent call last):
File "temp1.py", line 49, in <module>
tfidf_vectorizer.fit_transform(docList)
File "/usr/lib64/python2.6/site-packages/sklearn/feature_extraction/text.py", line 1285, in fit_transform
X = super(TfidfVectorizer, self).fit_transform(raw_documents)
File "/usr/lib64/python2.6/site-packages/sklearn/feature_extraction/text.py", line 825, in fit_transform
max_features)
File "/usr/lib64/python2.6/site-packages/sklearn/feature_extraction/text.py", line 697, in _limit_features
dfs = _document_frequency(X)
File "/usr/lib64/python2.6/site-packages/sklearn/feature_extraction/text.py", line 491, in _document_frequency
return bincount(X.indices, minlength=X.shape[1])
File "/usr/lib64/python2.6/site-packages/sklearn/utils/fixes.py", line 345, in bincount
return np.bincount(x, weights, minlength)
TypeError: function takes at most 2 arguments (3 given)
这是安装sklearn版本的问题吗?我的笔记本电脑安装了0.17.1,服务器安装了sklearn 0.16.1。由于我的语料库很大,我必须在服务器上运行它,否则我自然会遇到内存问题。
对此问题的任何见解将不胜感激。 谢谢:))
答案 0 :(得分:0)
我认为这与numpy的错误安装有关,报告为here。
您可以运行测试并检查(需要鼻子):
{{1}}
如果是这种情况,你需要删除numpy(也可能是scikit-learn)并使用推荐的说明重新安装它们:http://scikit-learn.org/stable/install.html
使用conda也是一个不错的选择:https://www.continuum.io/downloads