Question

我尝试使用tensorflow为keras（https://www.tensorflow.org/guide/keras/train_and_evaluate#custom_metrics）提供的自定义指标指南为我的项目实现cohen kappa指标。

在训练了模型并预测了输出之后，我将结果与sklearn.metrics.cohen_kappa_score进行了比较，结果并不相同。谁能说出解决此问题的方法？这是代码：

class BinaryKappa(keras.metrics.Metric):
    """Stateful Metric to calculate kappa over all batches.
    Assumes predictions and targets of shape `(samples, 1)`.
    # Arguments
        name: String, name for the metric.
    """

    def __init__(self, name='glo_kappa', **kwargs):
        super(BinaryKappa, self).__init__(name=name, **kwargs)
        self.true_positives = self.add_weight(name='tp',initializer='zeros')
        self.true_negative = self.add_weight(name='tn',initializer='zeros')
        self.false_positives = self.add_weight(name='fp',initializer='zeros')
        self.false_negative = self.add_weight(name='fn',initializer='zeros')

    def reset_states(self):
        self.true_positives.assign(0.)
        self.true_negative.assign(0.)
        self.false_positives.assign(0.)
        self.false_negative.assign(0.)

    def update_state(self, y_true, y_pred, sample_weight=None):
        y_true = K.cast(y_true, 'int32')
        y_pred = K.cast(K.round(y_pred), 'int32')
        true_pos = K.cast(K.sum(y_pred * y_true),'int32')
        true_neg = K.cast(K.sum((1 - y_pred) * (1 - y_true)),'int32')
        false_pos = K.cast(K.sum(y_pred * (1 - y_true)),'int32')
        false_neg = K.cast(K.sum((1 - y_pred) * y_true),'int32')

        true_pos = K.cast(true_pos, "float32")
        true_neg = K.cast(true_neg, "float32")
        false_pos = K.cast(false_pos, "float32")
        false_neg = K.cast(false_neg, "float32")

        self.true_positives.assign_add(true_pos)
        self.true_negative.assign_add(true_neg)
        self.false_positives.assign_add(false_pos)
        self.false_negative.assign_add(false_neg)

    def result(self):
        sm = self.true_positives + self.true_negative + self.false_positives + self.false_negative
        obs_agree = (self.true_positives + self.true_negative) / sm
        poss_pos = (self.true_positives + self.false_negative) * (self.true_positives + self.false_positives) / (sm**2)
        poss_neg = (self.true_negative + self.false_negative) * (self.true_negative + self.false_positives) / (sm**2)
        poss_agree = poss_pos + poss_neg
        return (obs_agree - poss_agree) / (1 - poss_agree + K.epsilon())

Answer 1

度量是离散的，因此您需要一个近似值。但是

看看tf.contrib，它实现了cohen_kappa。

https://github.com/tensorflow/tensorflow/blob/23c218785eac5bfe737eec4f8081fd0ef8e0684d/tensorflow/contrib/metrics/python/ops/metric_ops.py#L3595

甚至更好的是在kaggle.com上查看该指标的实现方式

https://www.kaggle.com/christofhenkel/weighted-kappa-loss-for-keras-tensorflow

Answer 2

使用confuse_matrix的信息来执行此操作，而不要注意其他的sklearn库。

从sklearn.metrics导入confusion_matrix

NBC = NBC.fit(X_train,y_train)
cm = confusion_matrix(y_test, NBC.predict(X_test))
tn, fp, fn, tp = cm.ravel()
print('tn: ',tn)
print('fp: ',fp)
print('fn: ',fn)
print('tp: ',tp)
print('------------------')
print(cm) 

p_0 = (tn+??)/(tn+fp+fn+??)
print('p_0:',p_0)

P_a = ((tn+fp)/(tn+fp+fn+??))*((tn+fn)/(tn+fp+fn+??))
print('P_yes: ',P_yes)

P_b = ((fn+??)/(tn+fp+fn+??))*((fp+??)/(tn+fp+fn+??))
print('P_no: ',P_no)


pe = P_a + P_b
print('pe: ',pe)

κ = (p_0-pe)/(1-pe)
print('κ: ',κ)

如何在喀拉拉邦正确实施科恩kappa度量标准？

2 个答案: