Question

如何为多类别Catboost分类器的每个类别找到F1分数？我已经读过documentation和github repo，那里有人问相同的问题。但是，我无法弄清楚如何实现此目的。我了解我必须在custom_metric中使用CatBoostClassifier()参数，但是当我想为多类的每个类别都获得custom_metric得分时，我不知道F1可以接受哪些参数数据集。

假设您有一个玩具数据集（来自文档）：

from catboost import Pool
cat_features = [0, 1, 2]
data = [["a","b", 1, 4, 5, 6],
        ["a","b", 4, 5, 6, 7],
        ["c","d", 30, 40, 50, 60]]

label = [0, 1, 2]

from sklearn.model_selection import train_test_split    
X_train, X_test, y_train, y_test = train_test_split(data, labels, test_size=0.2)
train_pool = Pool(X_train, y_train, cat_features=categorical_features_indices)
validate_pool = Pool(X_test, y_test, cat_features=categorical_features_indices)
params = {"loss_function": "MultiClass",
          "depth": symmetric_tree_depth,
          "num_trees": 500,
#           "eval_metric": "F1", # this doesn't work
          "verbose": False}

model = CatBoostClassifier(**params)
model.fit(train_pool, eval_set=validate_pool)

Answer 1

你应该使用 TotalF1

params = {
    'leaf_estimation_method': 'Gradient',
    'learning_rate': 0.01,
    'max_depth': 8,
    'bootstrap_type': 'Bernoulli',
    'objective': 'MultiClass',
    'eval_metric': 'MultiClass',
    'subsample': 0.8,
    'random_state': 42,
    'verbose': 0,
    "eval_metric" : 'TotalF1',
    "early_stopping_rounds" : 100
    }

https://catboost.ai/docs/concepts/loss-functions-multiclassification.html

Python Catboost：多类F1得分自定义指标

1 个答案: