请参阅下面的编辑，最初的帖子现在几乎没有任何意义，但问题仍然存在。
我开发了一个神经网络来对图像进行语义分割。我研究了各种损失函数（分类交叉熵 (CCE)、权重 CCE、焦点损失、tversky 损失、jaccard 损失、焦点 tversky 损失等），这些函数试图处理高度偏斜的类表示，但都没有产生预期的效果。我的顾问提到尝试创建一个自定义损失函数，该函数忽略特定类别的误报（但仍会惩罚误报）。
我有一个 6 类问题，我的网络设置为在单热编码的真实数据中工作/使用。因此，我的损失函数将接受形状为 y_true, y_pred（当前为 (batch, row, col, class)）的两个张量 (8, 128, 128, 6)。为了能够利用我已经探索过的损失，我想更改 y_pred 以将特定类别（第 0 个类别）的预测值设置为始终正确。这就是 y_true == class 0 设置 y_pred == class 0 的地方，否则什么都不做。
由于 tensorflow 张量是不可变的，我花了太多时间试图创建这个损失函数。我的第一次尝试（我是通过 numpy 的经验促成的）
def weighted_categorical_crossentropy_ignore(weights):
weights = K.variable(weights)
def loss(y_true, y_pred):
y_pred[tf.where(y_true == [1, 0, 0, 0, 0, 0])] = [1, 0, 0, 0, 0, 0]
# Scale predictions so that the class probs of each sample sum to 1
y_pred /= K.sum(y_pred, axis=-1, keepdims=True)
# Clip to prevent NaN's and Inf's
y_pred = K.clip(y_pred, K.epsilon(), 1 - K.epsilon())
loss = y_true * K.log(y_pred) * weights
loss = -K.sum(loss, -1)
return loss
return loss
虽然显然我不能改变 y_pred 所以这次尝试失败了。我最终创建了一些怪物，试图通过迭代 [batch, row, col] 并进行比较来“构建”张量。虽然这种（ese）尝试在技术上没有失败，但他们从未真正开始训练。我假设计算损失需要几分钟的时间。
经过多次失败的努力，我开始尝试在 SSCCE 中以纯 numpy 执行必要的计算。但是保持认知，我基本上仅限于实例化“简单”张量（即 ones、zeros）并且只执行“简单”操作，例如逐元素乘法、加法和整形。于是我来到了这个 SSCCE
import numpy as np
from tensorflow.keras.utils import to_categorical
# Generate the "images" at random
true_flat = np.argmax(np.random.rand(1, 2, 2, 4), axis=3).astype('int')
true = to_categorical(true_flat, num_classes=4).astype('int')
pred_flat = np.argmax(np.random.rand(1, 2, 2, 4), axis=3).astype('int')
pred = to_categorical(pred_flat, num_classes=4).astype('int')
print('True:\n', true_flat)
print('Pred:\n', pred_flat)
# Create a mask representing an all "class 0" image
class_zero_label = np.array([1, 0, 0, 0])
czl_all = class_zero_label * np.ones(true.shape).astype('int')
# Mask both the truth and pred to locate class 0 pixels
czl_true_locs = czl_all * true
czl_pred_locs = czl_all * pred
# Subtract to create "addition" matrix
a = (czl_true_locs - czl_pred_locs) * czl_true_locs
print('a:\n', a)
# Do this
m = ((a + 1) - (a * 2))
print('m - ', m.shape, ':\n', m)
# Pull the front entry from 'm' and "expand" its value
#x = (m[:, :, :, 0].flatten() * np.ones(pred.shape).astype('int')).T.reshape(pred.shape)
m_front = m[:, :, :, 0]
print('m_front - ', m_front.shape, ':\n', m_front)
#m_flat = m_front.flatten()
m_flat = m_front.reshape(m_front.shape[0], m_front.shape[1]*m_front.shape[2])
print('m_flat - ', m_flat.shape, ':\n', m_flat)
m_expand = m_flat * np.ones(pred.shape).astype('int')
print('m_expand - ', m_expand.shape, ':\n', m_expand)
m_trans = m_expand.T
m_fixT = m_trans.reshape(pred.shape)
print('m_fixT - ', m_fixT.shape, ':\n', m_fixT)
m = m_fixT
print('m:\n', m.shape)
# Perform the math as described
pred = (pred * m) + a
print('Pred:\n', np.argmax(pred, axis=3))
这个 SSCCE，很好，很糟糕，很复杂。基本上我的目标是创建两个矩阵，“加法”和“乘法”矩阵。乘法矩阵旨在将预测值中的每个像素“归零”，其中真值等于 0 类。这与像素值（即单热编码向量）将其归零是否等于 { {1}}。然后，加法矩阵旨在将向量 [0, 0, 0, 0, 0, 0] 添加到每个归零位置。最终这将实现将每个真正的 0 类像素的预测值设置为正确的目标。
问题是这个 SSCCE 没有完全转换为 tensorflow 操作。第一个问题是乘法矩阵的生成，当 [1, 0, 0, 0, 0, 0] 时没有正确定义。我想没关系，只是为了看看它是否有效，我会分解并batch_size > 1 tf.unstack 和 y_true 张量并对其进行迭代。这导致我对我的损失函数进行了当前的实例化
y_pred此损失函数的当前问题在于执行操作时 def weighted_categorical_crossentropy_ignore(weights):
weights = K.variable(weights)
def loss(y_true, y_pred):
y_true_un = tf.unstack(y_true)
y_pred_un = tf.unstack(y_pred)
y_pred_new = []
for i in range(0, y_true.shape[0]):
yt = y_true_un[i]
yp = y_pred_un[i]
# Pred:
# [[[0 3] * [[[1 0] + [[[0 1] = [[[0 0]
# [3 1]]] [[1 1]]] [[0 0]]] [[3 1]]]
# If we multiple pred by a tensor which zeros out only incorrect class 0 labelleling
# Then add class zero to those zero'd out locations
# We can negate the effect of mis-classified class 0 pixels but still punish for
# incorrectly predicted class 0 labels for other classes.
# Create a mask respresenting an all "class 0" image
class_zero_label = K.variable([1.0, 0.0, 0.0, 0.0, 0.0, 0.0])
czl_all = class_zero_label * K.ones(yt.shape)
# Mask both true and pred to locate class 0 pixels
czl_true = czl_all * yt
czl_pred = czl_all * yp
# Subtract to create "addition matrix"
a = czl_true - czl_pred
# Do this.
m = ((a + 1) - (a * 2.))
# And this.
x = K.flatten(m[:, :, 0])
x = x * K.ones(yp.shape)
x = K.transpose(x)
x = K.reshape(x, yp.shape)
# Voila.
ypnew = (yp * x) + a
y_pred_new.append(ypnew)
y_pred_new = tf.concat(y_pred_new, 0)
# Continue calculating weighted categorical crossentropy
# -------------------------------------------------------
# Scale predictions so that the class probs of each sample sum to 1
y_pred_new /= K.sum(y_pred_new, axis=-1, keepdims=True)
# Clip to prevent NaN's and Inf's
y_pred_new = K.clip(y_pred_new, K.epsilon(), 1 - K.epsilon())
loss = y_true * K.log(y_pred_new) * weights
loss = -K.sum(loss, -1)
return loss
return loss
和 numpy 之间的行为存在明显差异
tensorflow表示行为的含义
x = K.flatten(m[:, :, 0])
x = x * K.ones(yp.shape)
来自 SSCCE。
所以在这一点上，我觉得我已经深入研究了穴居人编码，我无法摆脱它。我必须想象有一些简单的方法类似于我最初尝试执行所描述的行为。
所以，我想我的直接问题是我该如何实施
m_flat = m_front.flatten()
m_expand = m_flat * np.ones(pred.shape).astype('int')
在自定义张量流损失函数中？
编辑：在摸索了很多之后，我终于确定了如何在 y_pred[tf.where(y_true == [1, 0, 0, 0, 0, 0])] = [1, 0, 0, 0, 0, 0]
、.numpy() 张量上调用 y_true 以利用 {{1 }} 操作（显然在程序开始时设置 y_pred “不起作用”。我不得不将 numpy 传递给 tf.compat.v1.enable_eager_execution）。
这让我基本上实现了概述的第一次尝试
run_eagerly=True虽然看起来通过调用 Model().compile(...)（或之后使用它）我显然已经“破坏”了通过网络的路径/流。基于尝试 def weighted_categorical_crossentropy_ignore(weights):
weights = K.variable(weights)
def loss(y_true, y_pred):
yp = y_pred.numpy()
yt = y_true.numpy()
yp[np.nonzero(np.all(yt == [1, 0, 0, 0, 0, 0], axis=3))] = [1, 0, 0, 0, 0, 0]
# Continue calculating weighted categorical crossentropy
# -------------------------------------------------------
# Scale predictions so that the class probs of each sample sum to 1
yp /= K.sum(yp, axis=-1, keepdims=True)
# Clip to prevent NaN's and Inf's
yp = K.clip(yp, K.epsilon(), 1 - K.epsilon())
loss = y_true * K.log(yp) * weights
loss = -K.sum(loss, -1)
return loss
return loss
y_pred.numpy()我假设我需要以某种方式将张量“重新编组”回 GPU 内存？我试过了
.fit无济于事；同样的错误。所以我想同样的问题仍然存在，但动机不同..
EDIT2：好吧，从这个 SO Answer 看来，我实际上不能使用 ValueError: No gradients provided for any variable: ['conv3d/kernel:0', <....>
来编组 yp = tf.convert_to_tensor(yp)
，numpy() 来使用普通的 y_true 操作。这必然会“破坏”网络路径，因此无法计算梯度。
结果我意识到使用 y_pred 我可以numpy 我的 run_eagerly=True/tf.Variable 并执行任务。所以在纯 tensorflow 中，我尝试再次重新创建相同的代码
y_true但是，这显然会产生与调用 y_pred 时相同的问题；无法计算梯度。所以我似乎又回到了第 1 方格。
EDIT3：使用建议的解决方案by gobrewers14 in the answer posted below，但根据我对问题的了解进行了修改，我产生了这个损失函数
def weighted_categorical_crossentropy_ignore(weights):
weights = K.variable(weights)
def loss(y_true, y_pred):
# yp = y_pred.numpy().copy()
# yt = y_true.numpy().copy()
# yp[np.nonzero(np.all(yt == [1, 0, 0, 0, 0, 0], axis=3))] = [1, 0, 0, 0, 0, 0]
yp = K.variable(y_pred)
yt = K.variable(y_true)
#np.all
x = K.all(yt == [1, 0, 0, 0, 0, 0], axis=3)
#np.nonzero
ne = tf.not_equal(x, tf.constant(False))
y = tf.where(ne)
# Perform the desired operation
yp[y] = [1, 0, 0, 0, 0, 0]
# Continue calculating weighted categorical crossentropy
# -------------------------------------------------------
# Scale predictions so that the class probs of each sample sum to 1
#yp /= K.sum(yp, axis=-1, keepdims=True) # Cannot use \= on tf.var, must use var = var /
yp = yp / K.sum(yp, axis=-1, keepdims=True)
# Clip to prevent NaN's and Inf's
yp = K.clip(yp, K.epsilon(), 1 - K.epsilon())
loss = y_true * K.log(yp) * weights
loss = -K.sum(loss, -1)
return loss
return loss
假设原始答案假定 .numpy() 的形状为 def weighted_categorical_crossentropy_ignore(weights):
weights = K.variable(weights)
def loss(y_true, y_pred):
print('y_true.shape: ', y_true.shape)
print('y_pred.shape: ', y_pred.shape)
# Generate modified y_pred where all truly class0 pixels are correct
y_true_class0_indicies = tf.where(tf.math.equal(y_true, [1., 0., 0., 0., 0., 0.]))
y_pred_updates = tf.repeat([
[1.0, 0.0, 0.0, 0.0, 0.0, 0.0]],
repeats=y_true_class0_indicies.shape[0],
axis=0)
yp = tf.tensor_scatter_nd_update(y_pred, y_true_class0_indicies, y_pred_updates)
# Continue calculating weighted categorical crossentropy
# -------------------------------------------------------
# Scale predictions so that the class probs of each sample sum to 1
yp /= K.sum(yp, axis=-1, keepdims=True)
# Clip to prevent NaN's and Inf's
yp = K.clip(yp, K.epsilon(), 1 - K.epsilon())
loss = y_true * K.log(yp) * weights
loss = -K.sum(loss, -1)
return loss
return loss
（即“平面”类表示，而不是单热编码表示 y_true），我首先打印[8, 128, 128] 和 [8, 128, 128, 6] 输入张量以确保完整性
y_true为了进一步理智，y_pred 的尾部提供的网络输出形状是
y_true.shape: (8, 128, 128, 6)
y_pred.shape: (8, 128, 128, 6)
然后我按照建议的解决方案中的“模式”，用 model.summary 替换原来的 conv2d_18 (Conv2D) (None, 128, 128, 6) 1542 dropout_5[0][0]
__________________________________________________________________________________________________
activation_9 (Activation) (None, 128, 128, 6) 0 conv2d_18[0][0]
==================================================================================================
Total params: 535,551,494
Trainable params: 535,529,478
Non-trainable params: 22,016
__________________________________________________________________________________________________
来处理 one-hot 编码的情况。根据我目前对提议的解决方案的理解（经过约 10 分钟的检查），我认为这应该可行。虽然在尝试训练模型时抛出以下异常
tf.math.equal(y_true, 0)因此，似乎（正如我给它们命名的那样）tf.math.equal(y_true, [1., 0., 0., 0., 0., 0.]) 的产生产生了一个具有“太多”元素的“折叠”张量。我了解使用 InvalidArgumentError: Inner dimensions of output shape must match inner dimensions of updates shape. Output: [8,128,128,6] updates: [684584,6] [Op:TensorScatterUpdate]
的动机，但其具体用途似乎不正确。我认为它应该根据我对 y_pred_updates 的理解生成形状为 tf.repeat 的张量。我认为这很可能只是基于在调用 (8, 128, 128, 6) 期间对 tf.tensor_scatter_nd_update 和 repeats 的选择。
答案 0 :(得分：1)
如果我正确理解您的问题，您正在寻找这样的东西：
import tensorflow as tf
# batch of true labels
y_true = tf.constant([5, 0, 1, 3, 4, 0, 2, 0], dtype=tf.int64)
# batch of class probabilities
y_pred = tf.constant(
[
[0.34670502, 0.04551039, 0.14020428, 0.14341979, 0.21430719, 0.10985339],
[0.25681055, 0.14013883, 0.19890164, 0.11124421, 0.14526634, 0.14763844],
[0.09199252, 0.21889475, 0.1170236 , 0.1929019 , 0.20311192, 0.17607528],
[0.3246354 , 0.23257554, 0.15549366, 0.17282239, 0.00000001, 0.11447308],
[0.16502093, 0.13163856, 0.14371352, 0.19880624, 0.23360236, 0.12721846],
[0.27362782, 0.21408406, 0.10917682, 0.13135742, 0.10814326, 0.16361059],
[0.20697299, 0.23721898, 0.06455399, 0.11071447, 0.18990229, 0.19063729],
[0.10320242, 0.22173141, 0.2547973 , 0.2314068 , 0.07063974, 0.11822232]
], dtype=tf.float32)
# find the indices in the batch where the true label is the class 0
indices = tf.where(tf.math.equal(y_true, 0))
# create a tensor with the number of updates you want to replace in `y_pred`
updates = tf.repeat(
[[1.0, 0.0, 0.0, 0.0, 0.0, 0.0]],
repeats=indices.shape[0],
axis=0)
# insert the updates into `y_pred` at the specified indices
modified_y_pred = tf.tensor_scatter_nd_update(y_pred, indices, updates)
print(modified_y_pred)
# tf.Tensor(
# [[0.34670502, 0.04551039, 0.14020428, 0.14341979, 0.21430719, 0.10985339],
# [1.00000000, 0.00000000, 0.00000000, 0.00000000, 0.00000000, 0.00000000],
# [0.09199252, 0.21889475, 0.1170236 , 0.1929019 , 0.20311192, 0.17607528],
# [0.3246354 , 0.23257554, 0.15549366, 0.17282239, 0.00000001, 0.11447308],
# [0.16502093, 0.13163856, 0.14371352, 0.19880624, 0.23360236, 0.12721846],
# [1.00000000, 0.00000000, 0.00000000, 0.00000000, 0.00000000, 0.00000000],
# [0.20697299, 0.23721898, 0.06455399, 0.11071447, 0.18990229, 0.19063729],
# [1.00000000, 0.00000000, 0.00000000, 0.00000000, 0.00000000, 0.00000000]],
# shape=(8, 6), dtype=tf.float32)
这个最终张量 modified_y_pred 可用于微分。
编辑：
使用面具可能更容易做到这一点。
示例：
# these arent normalized to 1 but you get the point
probs = tf.random.normal([2, 4, 4, 6])
# raw labels per pixel
labels = tf.random.uniform(
shape=[2, 4, 4],
minval=0,
maxval=6,
dtype=tf.int64)
# your labels are already one-hot encoded
labels = tf.one_hot(labels, 6)
# boolean mask where classes are `0`
# converting back to int labels with argmax for purposes of
# using `tf.math.equal`. Matching on `[1, 0, 0, 0, 0, 0]` is
# potentially buggy; matching on an integer is a lot more
# explicit.
mask = tf.math.equal(tf.math.argmax(labels, -1), 0)[..., None]
# flip the mask to zero out the pixels across channels where
# labels are zero
probs *= tf.cast(tf.math.logical_not(mask), tf.float32)
# multiply the mask by the one-hot labels, and add back
# to the already masked probabilities.
probs += labels * tf.cast(mask, tf.float32)