Question

我有一个功能联盟，它使用一些自定义变换器来选择文本和数据帧的一部分。我想了解它使用的功能。

管道选择并转换列，然后选择k best。我可以使用以下代码从k best中提取功能：

mask = union.named_steps['select_features'].get_support()

但是我无法将此蒙版应用于特征联合输出，因为我正在努力返回最终转换。我想我需要定义一个＆＃39; get_feature_names＆＃39;自定义变换器中的函数 - see related post。

管道如下：

union = Pipeline([
('feature_union', FeatureUnion([

    ('pipeline_1', Pipeline([
        ('selector', TextSelector(key='notes_1')),
        ('vectorise', CountVectorizer())
    ])),

    ('pipeline_2', Pipeline([
        ('selector', TextSelector(key='notes_2')),
        ('vectorise', CountVectorizer())
    ])),

    ('pipeline_3', Pipeline([
        ('selector', TextSelector(key='notes_3')),
        ('vectorise', CountVectorizer())
    ])),

    ('pipeline_4', Pipeline([
        ('selector', TextSelector(key='notes_4')),
        ('vectorise', CountVectorizer())
    ])),

    ('tf-idf_pipeline', Pipeline([
        ('selector', TextSelector(key='notes_5')),
        ('Tf-idf', TfidfVectorizer())
    ])),

    ('categorical_pipeline', Pipeline([
        ('selector', DataFrameSelector(['area', 'type', 'age'], True)),
        ('one_hot_encoding', OneHotEncoder(handle_unknown='ignore'))
    ]))
], n_jobs=-1)),
('select_features', SelectKBest(k='all')),
('classifier', MLPClassifier())
])

自定义变换器如下所示我已经尝试过包含一个＆＃39; get_feature_names＆＃39;每个变压器内的功能都没有正常工作：

class TextSelector(BaseEstimator, TransformerMixin):
   def __init__(self, key):
       self.key = key

   def fit(self, X, y=None):
       return self

   def transform(self, X):
       return X[self.key]

   def get_feature_names(self):
       return X[self.key].columns.tolist()


class DataFrameSelector(BaseEstimator, TransformerMixin):
   def __init__(self, attribute_names, factorize=False):
    self.attribute_names = attribute_names
    self.factorize = factorize

   def transform(self, X):
    selection = X[self.attribute_names]
    if self.factorize:
        selection = selection.apply(lambda p: pd.factorize(p)[0] + 1)
       return selection.values

   def fit(self, X, y=None):
       return self

   def get_feature_names(self):
       return X.columns.tolist()

感谢您的帮助。

Answer 1

这个对我有用。完全按照建议

union = Pipeline([
('feature_union', FeatureUnion([

('pipeline_1', MyPipeline([
    ('selector', TextSelector(key='notes_1')),
    ('vectorise', CountVectorizer())
])),
])

class myPipeline(Pipeline):
    def get_feature_names(self):
        for name, step in self.steps:
            if isinstance(step,TfidfVectorizer):
                return step.get_feature_names()

Answer 2

如果您知道步骤的名称（例如pipeline_1）和调用自定义转换器的子步骤的名称（例如vectorise），则可以直接参考这些步骤并按其名称划分子步骤

fnames = dict(union.named_steps['feature_union']
            .transformer_list)
            .get('pipeline_1')
            .named_steps['vectorise']
            .get_feature_names()

Source used

Answer 3

到目前为止，获取嵌套功能的最佳方法（感谢edesz）：

pipeline = Pipeline(steps=[
     ("union", FeatureUnion(
      transformer_list=[
        ("descriptor", Pipeline(steps=[
            ("selector", ItemSelector(column="Description")),
            ("tfidf", TfidfVectorizer(min_df=5, analyzer=u'word'))
        ]))
    ],...

pvect= dict(pipeline.named_steps['union'].transformer_list).get('descriptor').named_steps['tfidf']

然后您将TfidfVectorizer（）实例传递给另一个函数：

Show_most_informative_features(pvect,
           pipeline.named_steps['classifier'], n=MostIF)

从sklearn功能联合

3 个答案: