决策树的特征重要性提取(scikit-learn)

时间:2015-12-12 11:36:49

标签: python tree scikit-learn decision-tree feature-extraction

我一直在努力控制在我建模的决策树中使用的功能的重要性。我有兴趣发现在节点处选择的每个特征的权重以及术语本身。我的数据是一堆文件。 这是我的决策树代码,我修改了scikit的代码片段 - 学习提取(http://scikit-learn.org/stable/auto_examples/ensemble/plot_forest_importances.html):

from sklearn.feature_extraction.text import TfidfVectorizer

### Feature extraction
tfidf_vectorizer = TfidfVectorizer(stop_words=stopwords,
                                 use_idf=True, tokenizer=None, ngram_range=(1,2))#ngram_range=(1,0)

tfidf_matrix = tfidf_vectorizer.fit_transform(data[:, 1]) 
terms = tfidf_vectorizer.get_features_names()
### Define Decision Tree and fit
dtclf = DecisionTreeClassifier(random_state=1234)

dt = data.copy()

y = dt["label"]
X = tfidf_matrix

fitdt = dtclf.fit(X, y)

from sklearn.datasets import load_iris
from sklearn import tree

### Visualize Devision Tree

with open('data.dot', 'w') as file:
    tree.export_graphviz(dtclf, out_file = file, feature_names = terms)
file.close()

import subprocess
subprocess.call(['dot', '-Tpdf', 'data.dot', '-o' 'data.pdf'])

### Extract feature importance

importances = dtclf.feature_importances_

indices = np.argsort(importances)[::-1]

# Print the feature ranking
print('Feature Ranking:')

for f in range(tfidf_matrix.shape[1]):
    if importances[indices[f]] > 0:
        print("%d. feature %d (%f)" % (f + 1, indices[f], importances[indices[f]]))
        print ("feature name: ", terms[indices[f]])
  1. 我是否正确假设使用术语[indices [f]](这是特征术语向量)将打印用于在特定节点拆分树的实际特征术语?
  2. 用GraphViz可视化的决策树有例如X [30],我假设这是指特征术语的数值解释。如何提取术语本身,以便我可以验证我在#1中部署的过程?
  3. 更新了代码

    fitdt = dtclf.fit(X, y)
    with open(...):
    tree.export_graphviz(dtclf, out_file = file, feature_names = terms)
    

    提前致谢

1 个答案:

答案 0 :(得分:0)

对于第一个问题,您需要使用terms = tfidf_vectorizer.get_feature_names()从矢量图中获取要素名称。对于第二个问题,您可以使用export_graphviz致电feature_names = terms以获取变量的实际名称以显示在您的可视化中(查看export_graphviz的完整文档,了解其他许多选项这可能有助于改善您的可视化。