单标签火车设置产生多标签输出scikit - 学习一个与休息

时间:2015-08-14 09:51:35

标签: python scikit-learn text-classification multilabel-classification

我想知道是否有可能使用单一标签训练集来产生多标记输出。

使用修改后的scikit学习示例如下。 列车集包含许多句子,标记为伦敦或纽约。

目前,使用火车组的结果是伦敦或纽约甚至是句子,包括对两个城市的引用。

有没有办法让算法为包含伦敦和纽约的句子生成两个标签而不触及火车组?

import numpy as np
from sklearn.pipeline import Pipeline
from sklearn.feature_extraction.text import CountVectorizer
from sklearn.svm import LinearSVC
from sklearn.feature_extraction.text import TfidfTransformer
from sklearn.multiclass import OneVsRestClassifier
from sklearn import preprocessing

X_train = np.array(["new york is a hell of a town",
                    "new york was originally dutch",
                    "the big apple is great",
                    "new york is also called the big apple",
                    "nyc is nice",
                    "people abbreviate new york city as nyc",
                    "the capital of great britain is london",
                    "london is in the uk",
                    "london is in england",
                    "london is in great britain",
                    "it rains a lot in london",
                    "london hosts the british museum"])
y_train_text = [["new york"],["new york"],["new york"],["new york"],["new york"],
                ["new york"],["london"],["london"],["london"],["london"],
                ["london"],["london"]]

X_test = np.array(['nice day in nyc',
                   'welcome to london',
                   'london is rainy',
                   'it is raining in britian',
                   'it is raining in britian and the big apple',
                   'it is raining in britian and nyc',
                   'hello welcome to new york. enjoy it here and london too',
                    "new york is great and so is london",
                    "i like london better than new york"])
target_names = ['New York', 'London']

lb = preprocessing.LabelBinarizer()
Y = lb.fit_transform(y_train_text)

classifier = Pipeline([
    ('vectorizer', CountVectorizer()),
    ('tfidf', TfidfTransformer()),
    ('clf', OneVsRestClassifier(LinearSVC()))])

classifier.fit(X_train, Y)
predicted = classifier.predict(X_test)
all_labels = lb.inverse_transform(predicted)

for item, labels in zip(X_test, all_labels):
    print '%s => %s' % (item, ', '.join(labels))

0 个答案:

没有答案