如何使用Python绘制成本曲线

Question

这个问题was already asked a year ago on StackExchange/Stats，但被标记为题外话，没有答案就结束了。

结果是，我的问题是相同的：是否有Python（scikit-learn或其他）成本曲线实现，如Cost curves: An improved method for visualizing classifier performance 中所述？如果没有，我该如何实施它，同时考虑地面真相标签，预测和可能的误分类成本？

此方法绘制工作点上的性能（标准化的预期成本）（基于正确分类阳性样本的概率的概率成本函数）。

在正样本和负样本的误分类成本都等于1的情况下，性能与错误率相对应，而工作点就是示例来自正类别的概率。

Answer 1

我正在研究它，并且我认为我已经有了一个可行的实现。

import numpy as np
from sklearn.metrics import roc_curve
import matplotlib.pyplot as plt

# %% INPUTS

# C(-|+)
cost_fn = <a scalar value>
# C(+|-)
cost_fp = <a scalar value>

# Ground truth
truth = <a list of 0 (negative class) or 1 (positive class)>
# Predictions from a classifier
score = <a list of [0,1] class probabilities>

# %% OUTPUTS

# 1D-array of x-axis values (normalized PC)
pc = None
# list of lines as (slope, intercept)
lines = []
# lower envelope of the list of lines as a 1D-array of y-axis values (NEC)
lower_envelope = []
# area under the lower envelope (the smaller, the better)
area = None

# %% COMPUTATION

# points from the roc curve, because a point in the ROC space <=> a line in the cost space
roc_fpr, roc_tpr, _ = roc_curve(truth, score)

# compute the normalized p(+)*C(-|+)
thresholds = np.arange(0, 1.01, .01)
pc = (thresholds*cost_fn) / (thresholds*cost_fn + (1-thresholds)*cost_fp)

# compute a line in the cost space for each point in the roc space
for fpr, tpr in zip(roc_fpr, roc_tpr):
    slope = (1-tpr-fpr)
    intercept = fpr
    lines.append((slope, intercept))

# compute the lower envelope
for x_value in pc:
    y_value = min([slope*x_value+intercept for slope, intercept in lines])
    lower_envelope.append(max(0, y_value))
lower_envelope = np.array(lower_envelope)

# compute the area under the lower envelope using the composite trapezoidal rule
area = np.trapz(lower_envelope, pc)

# %% EXAMPLE OF PLOT

# display each line as a thin dashed line
for slope, intercept in lines:
    plt.plot(pc, slope*pc+intercept, color="grey", lw=1, linestyle="--")

# display the lower envelope as a thicker black line
plt.plot(pc, lower_envelope, color="black", lw=3, label="area={:.3f}".format(area))

# plot parameters
plt.xlim([0.0, 1.0])
plt.ylim([0.0, 1.05*max(lower_envelope)])
plt.xlabel("Probability Cost Function")
plt.ylabel("Normalized Expected Cost")
plt.title("Cost curve")
plt.legend(loc="lower right")

plt.show()

使用cost_fn=cost_fp=1，乳腺癌数据集和高斯朴素贝叶斯分类器得分的结果示例：