mlr_measures_classif.costs（预测类型=“概率”）

Question

对成本敏感的指标mlr_measures_classif.costs需要一种'response'预测类型。

msr("classif.costs")
#<MeasureClassifCosts:classif.costs>
#* Packages: -
#* Range: [-Inf, Inf]
#* Minimize: TRUE
#* Properties: requires_task
#* Predict type: response

即使将学习者的predict_type设置为'prob'，此度量也似乎有效：

# get a cost sensitive task
task = tsk("german_credit")

# cost matrix as given on the UCI page of the german credit data set
# https://archive.ics.uci.edu/ml/datasets/statlog+(german+credit+data)
costs = matrix(c(0, 5, 1, 0), nrow = 2)
dimnames(costs) = list(truth = task$class_names, predicted = task$class_names)
print(costs)

# mlr3 needs truth in columns, predictions in rows
costs = t(costs)

# create measure which calculates the absolute costs
m = msr("classif.costs", id = "german_credit_costs", costs = costs, normalize = FALSE)

# fit models and calculate costs
learner = lrn("classif.rpart", predict_type = "prob")
rr = resample(task, learner, rsmp("cv", folds = 3))
rr$aggregate(m)

#german_credit_costs 
#               341

为什么将predict_type设置为'prob'起作用？这是一个错误，还是该度量在内部将概率转换为类？我想将类别预测为正面或负面的阈值在内部设置为0.5？可以更改此阈值吗？

Answer 1

msr("classif.costs")使用混淆矩阵进行计算：https://github.com/mlr-org/mlr3/blob/master/R/MeasureClassifCosts.R

将predict_type设置为prob时，将生成阈值为0.5的混淆矩阵。要在重新采样后更改它：

pred = rr$predictions()
lapply(pred, function(x) x$set_threshold(0.1)) #arbitrary threshold

rr$aggregate(m)

将其更改回：

lapply(pred, function(x) x$set_threshold(0.5))
rr$aggregate(m)

R6活动绑定的“美”。