将最好的GridSearch分类器写入表中

Question

我发现并成功测试了以下脚本，该脚本将Pipeline和GridSearchCV应用于分类器选择。该脚本输出最佳分类器及其准确性。

import numpy as np
from sklearn import datasets
from sklearn.linear_model import LogisticRegression
from sklearn.ensemble import RandomForestClassifier
from sklearn.model_selection import GridSearchCV
from sklearn.pipeline import Pipeline

from sklearn import datasets
iris = datasets.load_iris()
X_train = iris.data
y_train = iris.target
X_test = iris.data[:10] # Augmenting test data
y_test = iris.target[:10] # Augmenting test data

#Create a pipeline
pipe = Pipeline([('classifier', LogisticRegression())])

# Create space of candidate learning algorithms and their hyperparameters
search_space = [{'classifier': [LogisticRegression()],
                 'classifier__penalty': ['l1', 'l2'],
                 'classifier__C': np.logspace(0, 4, 10)},
                {'classifier': [RandomForestClassifier()],
                 'classifier__n_estimators': [10, 100, 1000],
                 'classifier__max_features': [1, 2, 3]}]

# Create grid search 
clf = GridSearchCV(pipe, search_space, cv=5, verbose=0)

# Fit grid search
best_model = clf.fit(X_train, y_train)

print('Best training accuracy: %.3f' % best_model.best_score_)
print('Best estimator:', best_model.best_estimator_.get_params()['classifier'])
# Predict on test data with best params
y_pred = best_model.predict(X_test)
# Test data accuracy of model with best params
print(classification_report(y_test, y_pred, digits=4))
print('Test set accuracy score for best params: %.3f' % accuracy_score(y_test, y_pred))

from sklearn.metrics import precision_recall_fscore_support
print(precision_recall_fscore_support(y_test, y_pred, 
average='weighted'))

如何调整脚本以便它不仅输出最佳分类器，在我们的示例中是LogReg，还是在其他分类器中选择的最佳分类器？在上面，我也希望看到RandomForestClassifier()的输出。

理想是一种解决方案，其中显示每种算法的最佳分类器（LogReg，RandomForest，..），并将每个最佳分类器分类到表中。第一列或索引应为模型，precision_recall_fscore_support值在右侧为行。然后应该按F分数对表格进行排序。

PS：虽然脚本有效，但我还不确定管道中LogisticRegression()的功能是什么，因为它稍后会在搜索空间中定义。

解决方案（简化）：

from sklearn import datasets
iris = datasets.load_iris()
X_train = iris.data
y_train = iris.target
X_test = iris.data[:10]
y_test = iris.target[:10]

seed=1
models = [
            'RFC',
            'logisticRegression'
         ]
clfs = [
        RandomForestClassifier(random_state=seed,n_jobs=-1),
        LogisticRegression()
        ]

params = {
            models[0]:{'n_estimators':[100]},
            models[1]: {'C':[1000]}
         }


for name, estimator in zip(models,clfs):

    print(name)

    clf = GridSearchCV(estimator, params[name], scoring='accuracy', refit='True', n_jobs=-1, cv=5)

    clf.fit(X_train, y_train)

    print("best params: " + str(clf.best_params_))
    print("best scores: " + str(clf.best_score_))

    y_pred = clf.predict(X_test)
    acc = accuracy_score(y_test, y_pred)

    print("Accuracy: {:.4%}".format(acc))
    print(classification_report(y_test, y_pred, digits=4))

Answer 1

如果我理解正确，这应该可以正常工作。

import pandas as pd
import numpy as np

df = pd.DataFrame(list(best_model.cv_results_['params']))
ranking = best_model.cv_results_['rank_test_score']
# The sorting is done based on the test_score of the models.
sorting = np.argsort(best_model.cv_results_['rank_test_score'])

# Sort the lines based on the ranking of the models
df_final = df.iloc[sorting]

# The first line contains the best model and its parameters
df_final.to_csv('sorted_table.csv')

# OR to avoid the index in the writting 
df_final.to_csv('sorted_table2.csv',index=False)

<强>结果：

但是，在这种情况下，订购不是基于F值完成的。为此，请使用此功能。在GridSearch scoring属性中定义f1_weighted并重复我的代码。

示例：

...
clf = GridSearchCV(pipe, search_space, cv=5, verbose=0,scoring='f1_weighted')

best_model = clf.fit(X_train, y_train)

df = pd.DataFrame(list(best_model.cv_results_['params']))
ranking = best_model.cv_results_['rank_test_score']

# The sorting is done based on the F values of the models.
sorting = np.argsort(best_model.cv_results_['rank_test_score'])

# Sort the lines based on the ranking of the models
df_final = df.iloc[sorting]

df_final.to_csv('F_sorted_table.csv')

<强>结果：