Question

我正在训练用于分类的神经网络。在我的研究背景下，我想将每个小批量中的（k）个最高损失归零。在某种程度上不依赖numpy的情况下，我无法找出执行此过程的简单方法。

我尝试了以下过程：
1.计算损失数组的argmax索引-返回tf张量
2.用指标数组切片损失张量

问题在于无法使用tf张量执行切片。

# losses is tf.Tensor
ind_sorted = tf.argsort(losses)
losses_sorted = losses[ind_sorted] # Error mentioned above
# The issue is that ind_1_sorted depends on the output of the neural network. I couldn't find an equivalent of the detach method in pytorch

k_smallest_losses = losses_sorted[:k] # Keeping only the k smallest losses
loss = tf.sum(k_smallest_losses) # Performing the summation of the k smallest losses

Answer 1

可能您想使用tf.nn.top_k，它会返回top_k项目的值和索引。（请注意，为了获得最小的损失，我会在您的损失上加上负数，并在完成后将其转换回去。）

batch = 2
max_len = 6
losses = tf.random.uniform(shape=[batch, max_len], minval=0, maxval=2, dtype = tf.float32)
bottom_losses_values, bottom_losses_indices = tf.nn.top_k(-losses, k=3)
total = tf.reduce_sum(-bottom_losses_values, axis=-1)
with tf.Session() as sess:
  losses, bottom_losses_values, bottom_losses_indices, total = sess.run([losses, bottom_losses_values, bottom_losses_indices, total])
  print 'original losses\n', losses
  print 'bottom 3 loss values\n', -bottom_losses_values
  print 'bottom 3 loss indices\n', bottom_losses_indices
  print 'total\n', total

结果：

original losses
[[ 1.45301318  1.65069246  1.31003475  1.71488905  1.71400714  0.0543921 ]
 [ 0.09954047  0.12081003  0.24793792  1.51561213  1.73758292  1.43859148]]
bottom 3 loss values
[[ 0.0543921   1.31003475  1.45301318]
 [ 0.09954047  0.12081003  0.24793792]]
bottom 3 loss indices
[[5 2 0]
 [0 1 2]]
total
[ 2.81744003  0.46828842]

Tensorflow-是否有一种简单的方法可以将小批量中损失最大的样品的损失归零？

1 个答案: