Question

我有一个问题，我尝试建立自己的分类器，它的完成，它的工作完美，但当我尝试使用交叉验证分数时，我得到一个错误：

  File "/home/webinterpret/workspace/nlp/wi-item-attribute-extraction/attr_extractor.py", line 95, in fit
    print self.fitted_models[attr][len(self.fitted_models[attr]) - 1].cross_validation_score(x_train, y_train, 5, 0.2)
  File "/home/webinterpret/workspace/nlp/wi-item-attribute-extraction/attr_extractor.py", line 163, in cross_validation_score
    cv=self.cv).mean()
  File "/home/webinterpret/.virtualenvs/nlp/local/lib/python2.7/site-packages/sklearn/cross_validation.py", line 1361, in cross_val_score
    for train, test in cv)
  File "/home/webinterpret/.virtualenvs/nlp/local/lib/python2.7/site-packages/sklearn/externals/joblib/parallel.py", line 659, in __call__
    self.dispatch(function, args, kwargs)
  File "/home/webinterpret/.virtualenvs/nlp/local/lib/python2.7/site-packages/sklearn/externals/joblib/parallel.py", line 406, in dispatch
    job = ImmediateApply(func, args, kwargs)
  File "/home/webinterpret/.virtualenvs/nlp/local/lib/python2.7/site-packages/sklearn/externals/joblib/parallel.py", line 140, in __init__
    self.results = func(*args, **kwargs)
  File "/home/webinterpret/.virtualenvs/nlp/local/lib/python2.7/site-packages/sklearn/cross_validation.py", line 1478, in _fit_and_score
    test_score = _score(estimator, X_test, y_test, scorer)
  File "/home/webinterpret/.virtualenvs/nlp/local/lib/python2.7/site-packages/sklearn/cross_validation.py", line 1534, in _score
    score = scorer(estimator, X_test, y_test)
  File "/home/webinterpret/.virtualenvs/nlp/local/lib/python2.7/site-packages/sklearn/metrics/scorer.py", line 201, in _passthrough_scorer
    return estimator.score(*args, **kwargs)
  File "/home/webinterpret/workspace/nlp/wi-item-attribute-extraction/attr_extractor.py", line 198, in score
    return (pd.Series(self.predict(x_test)) == y_test).mean()
  File "/home/webinterpret/workspace/nlp/wi-item-attribute-extraction/attr_extractor.py", line 190, in predict
    result[i] = 1 if self.pattern in item else 0
  File "/home/webinterpret/.virtualenvs/nlp/local/lib/python2.7/site-packages/scipy/sparse/compressed.py", line 216, in __eq__
    if np.isnan(other):
TypeError: Not implemented for this type

我的预测功能：

result = np.zeros(text.shape[0])
i = 0
for item in text:
    result[i] = 1 if self.pattern in item else 0
    i+=1
return result

错误在＆＃34;如果self.pattern在项目0和＃34;但我不知道如何以不同的方式制作它？

模式是一个文字，如：＆＃34; car＆＃34;，文字只是一个文字：＆＃34;这辆车坏了。＆＃34;

Answer 1

因此，scikit-learn真的非常希望您的数据采用严格的矩阵形式。 x_train应该是数字矩阵，y_train应该是数字的矩阵或向量。交叉验证例程对您的输入进行数组 - 以确保其内置分类器的格式正确。

在这里，正在发生的事情是array-ify步骤创建一个字符矩阵（有效），其中包含与最大长度文本一样多的列。因此，大多数文本行都使用＆＃34; np.nans＆＃34;来填充剩余的列。

如果您想使用这样的分类器，则需要避免使用内置的Pipeline和Cross Validation例程。您可以迭代交叉验证并构建自己的分数，如下所示：

for train,test in StratifiedKFold( target_classes ):
    train_data = data[train]
    test_data = data[test]
    # Train with train, predict with test, score with your favorite scorer

Python sklearn预测函数

1 个答案: