Question

是否可以使用sklearn指标作为tf.estimator中的自定义指标？我已经尝试过以下自定义评分功能。

from sklearn.metrics import recall_score
def my_score(labels, predictions):
    return {'MARecall': recall_score(labels, predictions, average='macro')}

但是它不起作用：

eval_results = classifier.evaluate(input_fn=tf.estimator.inputs.numpy_input_fn(x={'x': val_x}, y=np.array(val_y), num_epochs=1, batch_size=20, shuffle=False))

... ...
... ...
<ipython-input-81-e433b0457af2> in my_acc(labels, predictions)
      1 def my_acc(labels, predictions):
----> 2     return {'WA': np.array(recall_score(labels, predictions, average='micro'))}
      3

/anaconda/envs/py35/lib/python3.5/site-packages/sklearn/metrics/classification.py in recall_score(y_true, y_pred, labels, pos_label, average, sample_weight)
   1357                                                  average=average,
   1358                                                  warn_for=('recall',),
-> 1359                                                  sample_weight=sample_weight)
   1360     return r
   1361

/anaconda/envs/py35/lib/python3.5/site-packages/sklearn/metrics/classification.py in precision_recall_fscore_support(y_true, y_pred, beta, labels, pos_label, average, warn_for, sample_weight)
   1023         raise ValueError("beta should be >0 in the F-beta score")
   1024
-> 1025     y_type, y_true, y_pred = _check_targets(y_true, y_pred)
   1026     present_labels = unique_labels(y_true, y_pred)
   1027

/anaconda/envs/py35/lib/python3.5/site-packages/sklearn/metrics/classification.py in _check_targets(y_true, y_pred)
     70     """
     71     check_consistent_length(y_true, y_pred)
---> 72     type_true = type_of_target(y_true)
     73     type_pred = type_of_target(y_pred)
     74

/anaconda/envs/py35/lib/python3.5/site-packages/sklearn/utils/multiclass.py in type_of_target(y)
    242     if not valid:
    243         raise ValueError('Expected array-like (array or non-string sequence), '
--> 244                          'got %r' % y)
    245
    246     sparseseries = (y.__class__.__name__ == 'SparseSeries')

ValueError: Expected array-like (array or non-string sequence), got <tf.Tensor 'fifo_queue_DequeueUpTo:2' shape=(?,) dtype=int64>

有没有解决的办法？我手头有一个多类别的分类问题，需要在训练和评估期间同时记录宏观和微观平均分数。

Answer 1

为了在tensorflow训练循环中正常工作，需要更新指标。 tf.metrics中的每个函数都有一个update_op。因此，强烈建议您使用tf.metrics.true_positives等较低级别的函数来构建自定义指标，我不知道sklearn中使用的特定公式，您可以像这样定义自己的指标

def custom_metric(labels, predictions):
    metric1, update_op_m1 = tf.metrics.any_function(labels, predictions)
    metric2, update_op_m2 = tf.metrics.any_other_function(labels, predictions)
    output = tf.reduce_mean(metric1 + metric2)
    return output, tf.group(update_op_m1, update_op_m2) #Note that you need to group all update ops

Answer 2

一种解决方案是使用https://github.com/guillaumegenthial/tf_metrics

中的度量函数

tensorflow估算器自定义指标：使用sklearn指标吗？

2 个答案: