我正在使用带有自定义评估功能的xgboost，并且我想实施“提前停止”设置150轮的限制。
我得到的评估指标比预期的2个多，并且我不知道如何解释它们。此外，我不确定如何激活提前停止设置限制（例如150发）。
有关可重现的示例：
import numpy as np
def F1_eval_gen(preds, labels):
t = np.arange(0, 1, 0.005)
f = np.repeat(0, 200)
results = np.vstack([t, f]).T
# assuming labels only containing 0's and 1's
n_pos_examples = sum(labels)
if n_pos_examples == 0:
n_pos_examples = 1
for i in range(200):
pred_indexes = (preds >= results[i, 0])
TP = sum(labels[pred_indexes])
FP = len(labels[pred_indexes]) - TP
precision = 0
recall = TP / n_pos_examples
if (FP + TP) > 0:
precision = TP / (FP + TP)
if (precision + recall > 0):
F1 = 2 * precision * recall / (precision + recall)
else:
F1 = 0
results[i, 1] = F1
return (max(results[:, 1]))
def F1_eval(preds, dtrain):
res = F1_eval_gen(preds, dtrain.get_label())
return 'f1_err', 1-res
from sklearn import datasets
from sklearn.model_selection import *
skl_data = datasets.load_breast_cancer()
X = skl_data.data
y = skl_data.target
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size = 0.2)
scale_pos_weight = sum(y_train == 0)/sum(y_train == 1)
base_score = sum(y_train == 1)/len(y_train)
max_depth = 6
learning_rate = 0.1
gamma = 0
min_child_weight = 1
subsample = 0.8
colsample_bytree = 0.8
colsample_bylevel = 1
reg_alpha = 0
reg_lambda = 1
clf = xgb.XGBClassifier(max_depth= max_depth, learning_rate= learning_rate,silent=False, objective='binary:logistic', \
booster='gbtree', n_jobs=8, nthread=None, gamma=gamma, min_child_weight=min_child_weight, max_delta_step=0, \
subsample= subsample, colsample_bytree=colsample_bytree, colsample_bylevel=colsample_bylevel, \
reg_alpha= reg_alpha, reg_lambda=1, scale_pos_weight= scale_pos_weight, base_score= base_score)
clf.fit(X_train, y_train,
eval_set=[(X_train, y_train), (X_test, y_test)], eval_metric= F1_eval, verbose=True)
..................
[94] validation_0-error:0 validation_1-error:0.035088 validation_0-f1_err:0 validation_1-f1_err:0.018634
[95] validation_0-error:0 validation_1-error:0.035088 validation_0-f1_err:0 validation_1-f1_err:0.018634
[96] validation_0-error:0 validation_1-error:0.035088 validation_0-f1_err:0 validation_1-f1_err:0.018634
[97] validation_0-error:0 validation_1-error:0.035088 validation_0-f1_err:0 validation_1-f1_err:0.018634
[98] validation_0-error:0 validation_1-error:0.035088 validation_0-f1_err:0 validation_1-f1_err:0.018634
[99] validation_0-error:0 validation_1-error:0.035088 validation_0-f1_err:0 validation_1-f1_err:0.018634
您的建议将不胜感激。
clf = xgb.XGBClassifier(max_depth= max_depth, niterations = 1000, learning_rate= learning_rate,silent=False, \
objective='binary:logistic', booster='gbtree', n_jobs=8, nthread=None, gamma=gamma,\
min_child_weight=min_child_weight, max_delta_step=0, \
subsample= subsample, colsample_bytree=colsample_bytree, colsample_bylevel=colsample_bylevel, \
reg_alpha= reg_alpha, reg_lambda=1, scale_pos_weight= scale_pos_weight, base_score= base_score)
clf.fit(X_train, y_train, early_stopping_rounds= 25,
eval_set=[(X_train, y_train), (X_test, y_test)], eval_metric= F1_eval, verbose=True)
[0] validation_0-error:0.386813 validation_1-error:0.315789 validation_0-f1_err:0.032609 validation_1-f1_err:0.031847
Multiple eval metrics have been passed: 'validation_1-f1_err' will be used for early stopping.
Will train until validation_1-f1_err hasn't improved in 25 rounds.
[1] validation_0-error:0.131868 validation_1-error:0.078947 validation_0-f1_err:0.016216 validation_1-f1_err:0.031056
[2] validation_0-error:0.048352 validation_1-error:0.052632 validation_0-f1_err:0.012522 validation_1-f1_err:0.037037
[3] validation_0-error:0.032967 validation_1-error:0.04386 validation_0-f1_err:0.008977 validation_1-f1_err:0.031447
[4] validation_0-error:0.01978 validation_1-error:0.04386 validation_0-f1_err:0.010753 validation_1-f1_err:0.031447
[5] validation_0-error:0.015385 validation_1-error:0.035088 validation_0-f1_err:0.008977 validation_1-f1_err:0.025316
[6] validation_0-error:0.013187 validation_1-error:0.04386 validation_0-f1_err:0.010676 validation_1-f1_err:0.025316
[7] validation_0-error:0.017582 validation_1-error:0.04386 validation_0-f1_err:0.010638 validation_1-f1_err:0.018868
[8] validation_0-error:0.013187 validation_1-error:0.04386 validation_0-f1_err:0.008913 validation_1-f1_err:0.025
[9] validation_0-error:0.008791 validation_1-error:0.04386 validation_0-f1_err:0.007143 validation_1-f1_err:0.025
[10] validation_0-error:0.010989 validation_1-error:0.04386 validation_0-f1_err:0.007143 validation_1-f1_err:0.025
[11] validation_0-error:0.008791 validation_1-error:0.04386 validation_0-f1_err:0.007143 validation_1-f1_err:0.025
[12] validation_0-error:0.008791 validation_1-error:0.052632 validation_0-f1_err:0.007143 validation_1-f1_err:0.025
[13] validation_0-error:0.008791 validation_1-error:0.052632 validation_0-f1_err:0.007117 validation_1-f1_err:0.025
[14] validation_0-error:0.008791 validation_1-error:0.052632 validation_0-f1_err:0.005348 validation_1-f1_err:0.018868
[15] validation_0-error:0.008791 validation_1-error:0.052632 validation_0-f1_err:0.005348 validation_1-f1_err:0.018868
[16] validation_0-error:0.008791 validation_1-error:0.052632 validation_0-f1_err:0.005348 validation_1-f1_err:0.018868
[17] validation_0-error:0.008791 validation_1-error:0.052632 validation_0-f1_err:0.005348 validation_1-f1_err:0.018868
[18] validation_0-error:0.008791 validation_1-error:0.052632 validation_0-f1_err:0.005348 validation_1-f1_err:0.018868
[19] validation_0-error:0.008791 validation_1-error:0.052632 validation_0-f1_err:0.005348 validation_1-f1_err:0.018868
[20] validation_0-error:0.008791 validation_1-error:0.052632 validation_0-f1_err:0.005348 validation_1-f1_err:0.018868
[21] validation_0-error:0.006593 validation_1-error:0.052632 validation_0-f1_err:0.005348 validation_1-f1_err:0.018868
[22] validation_0-error:0.006593 validation_1-error:0.052632 validation_0-f1_err:0.003571 validation_1-f1_err:0.018868
[23] validation_0-error:0.006593 validation_1-error:0.052632 validation_0-f1_err:0.003571 validation_1-f1_err:0.018868
[24] validation_0-error:0.006593 validation_1-error:0.052632 validation_0-f1_err:0.003571 validation_1-f1_err:0.018868
[25] validation_0-error:0.006593 validation_1-error:0.052632 validation_0-f1_err:0.003571 validation_1-f1_err:0.018868
[26] validation_0-error:0.004396 validation_1-error:0.052632 validation_0-f1_err:0.003571 validation_1-f1_err:0.018868
[27] validation_0-error:0.004396 validation_1-error:0.052632 validation_0-f1_err:0.003584 validation_1-f1_err:0.018868
[28] validation_0-error:0.004396 validation_1-error:0.052632 validation_0-f1_err:0.003584 validation_1-f1_err:0.018868
[29] validation_0-error:0.004396 validation_1-error:0.052632 validation_0-f1_err:0.003571 validation_1-f1_err:0.018868
[30] validation_0-error:0.004396 validation_1-error:0.052632 validation_0-f1_err:0.001789 validation_1-f1_err:0.018868
[31] validation_0-error:0.004396 validation_1-error:0.052632 validation_0-f1_err:0.001789 validation_1-f1_err:0.018868
[32] validation_0-error:0.004396 validation_1-error:0.052632 validation_0-f1_err:0.001789 validation_1-f1_err:0.018868
Stopping. Best iteration:
[7] validation_0-error:0.017582 validation_1-error:0.04386 validation_0-f1_err:0.010638 validation_1-f1_err:0.018868
XGBClassifier(base_score=0.6131868131868132, booster='gbtree',
colsample_bylevel=1, colsample_bytree=0.8, gamma=0,
learning_rate=0.1, max_delta_step=0, max_depth=6,
min_child_weight=1, missing=None, n_estimators=100, n_jobs=8,
niterations=1000, nthread=None, objective='binary:logistic',
random_state=0, reg_alpha=0, reg_lambda=1,
scale_pos_weight=0.6308243727598566, seed=None, silent=False,
subsample=0.8)
答案 0 :(得分：0)
您将获得4个评估矩阵，因为xgboost以某种方式向您的eval_set添加了另一个评估指标。就个人而言，我使用的是核心xgboost，而不是scikit。因此，如果您想了解更多信息，请在文档中阅读。
要提前停止，您必须在n_estimators=1000
xgb.XGBClassifier（或所需的迭代次数）设置为参数。
然后在early_stopping_rounds=50中设置clf.fit（或您想要的值）。
Here's the documentation。
提前停止将决定何时需要停止增强算法以避免过度拟合。这样做是通过评估您在tuple中定义的(X_test, y_test) eval_set来实现的。如果评估误差在 50 次迭代中没有减少，则early_stopping将停止提升您的实力。