我正在尝试使用scikit-learn的监督学习方法之一将文本片段分类为一个或多个类别。我尝试的所有算法的预测函数只返回一个匹配。
例如，我有一段文字：
"Theaters in New York compared to those in London"
我已经训练了算法为我提供的每个文本片段选择一个地方。
在上面的示例中，我希望它返回New York和London，但它只返回New York。
是否可以使用scikit-learn返回多个结果？或者甚至以最高概率返回标签？
感谢您的帮助。
---更新
我尝试使用OneVsRestClassifier，但我仍然只能在每段文字中找到一个选项。以下是我正在使用的示例代码
y_train = ('New York','London')
train_set = ("new york nyc big apple", "london uk great britain")
vocab = {'new york' :0,'nyc':1,'big apple':2,'london' : 3, 'uk': 4, 'great britain' : 5}
count = CountVectorizer(analyzer=WordNGramAnalyzer(min_n=1, max_n=2),vocabulary=vocab)
test_set = ('nice day in nyc','london town','hello welcome to the big apple. enjoy it here and london too')
X_vectorized = count.transform(train_set).todense()
smatrix2 = count.transform(test_set).todense()
base_clf = MultinomialNB(alpha=1)
clf = OneVsRestClassifier(base_clf).fit(X_vectorized, y_train)
Y_pred = clf.predict(smatrix2)
print Y_pred
结果：['纽约''伦敦''伦敦']
答案 0 :(得分：107)
您想要的是多标签分类。 Scikits-learn可以做到这一点。见这里：http://scikit-learn.org/dev/modules/multiclass.html。
我不确定你的例子中出了什么问题，我的sklearn版本显然没有WordNGramAnalyzer。也许这是一个使用更多训练样例或尝试不同分类器的问题？虽然请注意多标签分类器希望目标是元组/标签列表的列表。
以下适用于我：
import numpy as np
from sklearn.pipeline import Pipeline
from sklearn.feature_extraction.text import CountVectorizer
from sklearn.svm import LinearSVC
from sklearn.feature_extraction.text import TfidfTransformer
from sklearn.multiclass import OneVsRestClassifier
X_train = np.array(["new york is a hell of a town",
"new york was originally dutch",
"the big apple is great",
"new york is also called the big apple",
"nyc is nice",
"people abbreviate new york city as nyc",
"the capital of great britain is london",
"london is in the uk",
"london is in england",
"london is in great britain",
"it rains a lot in london",
"london hosts the british museum",
"new york is great and so is london",
"i like london better than new york"])
y_train = [[0],[0],[0],[0],[0],[0],[1],[1],[1],[1],[1],[1],[0,1],[0,1]]
X_test = np.array(['nice day in nyc',
'welcome to london',
'hello welcome to new york. enjoy it here and london too'])
target_names = ['New York', 'London']
classifier = Pipeline([
('vectorizer', CountVectorizer(min_n=1,max_n=2)),
('tfidf', TfidfTransformer()),
('clf', OneVsRestClassifier(LinearSVC()))])
classifier.fit(X_train, y_train)
predicted = classifier.predict(X_test)
for item, labels in zip(X_test, predicted):
print '%s => %s' % (item, ', '.join(target_names[x] for x in labels))
对我来说，这会产生输出：
nice day in nyc => New York
welcome to london => London
hello welcome to new york. enjoy it here and london too => New York, London
希望这有帮助。
答案 1 :(得分：57)
编辑：根据建议使用MultiLabelBinarizer更新了Python 3，scikit-learn 0.18.1。
我一直在研究这个问题，并对mwv的优秀答案进行了一些改进，这些答案可能很有用。它将文本标签作为输入而不是二进制标签，并使用MultiLabelBinarizer对其进行编码。
import numpy as np
from sklearn.pipeline import Pipeline
from sklearn.feature_extraction.text import CountVectorizer
from sklearn.svm import LinearSVC
from sklearn.feature_extraction.text import TfidfTransformer
from sklearn.multiclass import OneVsRestClassifier
from sklearn.preprocessing import MultiLabelBinarizer
X_train = np.array(["new york is a hell of a town",
"new york was originally dutch",
"the big apple is great",
"new york is also called the big apple",
"nyc is nice",
"people abbreviate new york city as nyc",
"the capital of great britain is london",
"london is in the uk",
"london is in england",
"london is in great britain",
"it rains a lot in london",
"london hosts the british museum",
"new york is great and so is london",
"i like london better than new york"])
y_train_text = [["new york"],["new york"],["new york"],["new york"],["new york"],
["new york"],["london"],["london"],["london"],["london"],
["london"],["london"],["new york","london"],["new york","london"]]
X_test = np.array(['nice day in nyc',
'welcome to london',
'london is rainy',
'it is raining in britian',
'it is raining in britian and the big apple',
'it is raining in britian and nyc',
'hello welcome to new york. enjoy it here and london too'])
target_names = ['New York', 'London']
mlb = MultiLabelBinarizer()
Y = mlb.fit_transform(y_train_text)
classifier = Pipeline([
('vectorizer', CountVectorizer()),
('tfidf', TfidfTransformer()),
('clf', OneVsRestClassifier(LinearSVC()))])
classifier.fit(X_train, Y)
predicted = classifier.predict(X_test)
all_labels = mlb.inverse_transform(predicted)
for item, labels in zip(X_test, all_labels):
print('{0} => {1}'.format(item, ', '.join(labels)))
这给了我以下输出：
nice day in nyc => new york
welcome to london => london
london is rainy => london
it is raining in britian => london
it is raining in britian and the big apple => new york
it is raining in britian and nyc => london, new york
hello welcome to new york. enjoy it here and london too => london, new york
答案 2 :(得分：7)
我也遇到了这个问题，对我来说问题是我的y_Train是一系列字符串，而不是一系列字符串序列。显然，OneVsRestClassifier将根据输入标签格式决定是使用多类还是多标签。所以改变：
y_train = ('New York','London')
到
y_train = (['New York'],['London'])
显然这将在将来消失，因为所有标签的断开都是相同的：https://github.com/scikit-learn/scikit-learn/pull/1987
答案 3 :(得分：7)
更改此行以使其在新版本的python中运行
# lb = preprocessing.LabelBinarizer()
lb = preprocessing.MultiLabelBinarizer()
答案 4 :(得分：2)
很少有多个分类示例如下： -
示例1： -
import numpy as np
from sklearn.preprocessing import LabelBinarizer
encoder = LabelBinarizer()
arr2d = np.array([1, 2, 3,4,5,6,7,8,9,10,11,12,13,14,1])
transfomed_label = encoder.fit_transform(arr2d)
print(transfomed_label)
输出
[[1 0 0 0 0 0 0 0 0 0 0 0 0 0]
[0 1 0 0 0 0 0 0 0 0 0 0 0 0]
[0 0 1 0 0 0 0 0 0 0 0 0 0 0]
[0 0 0 1 0 0 0 0 0 0 0 0 0 0]
[0 0 0 0 1 0 0 0 0 0 0 0 0 0]
[0 0 0 0 0 1 0 0 0 0 0 0 0 0]
[0 0 0 0 0 0 1 0 0 0 0 0 0 0]
[0 0 0 0 0 0 0 1 0 0 0 0 0 0]
[0 0 0 0 0 0 0 0 1 0 0 0 0 0]
[0 0 0 0 0 0 0 0 0 1 0 0 0 0]
[0 0 0 0 0 0 0 0 0 0 1 0 0 0]
[0 0 0 0 0 0 0 0 0 0 0 1 0 0]
[0 0 0 0 0 0 0 0 0 0 0 0 1 0]
[0 0 0 0 0 0 0 0 0 0 0 0 0 1]
[1 0 0 0 0 0 0 0 0 0 0 0 0 0]]
示例2： -
import numpy as np
from sklearn.preprocessing import LabelBinarizer
encoder = LabelBinarizer()
arr2d = np.array(['Leopard','Lion','Tiger', 'Lion'])
transfomed_label = encoder.fit_transform(arr2d)
print(transfomed_label)
输出
[[1 0 0]
[0 1 0]
[0 0 1]
[0 1 0]]