Question

我正在尝试实现本文所述的内核预测技术：

http://cvc.ucsb.edu/graphics/Papers/SIGGRAPH2017_KPCN/

总而言之，我相信我需要将网络的输出设置为KxK内核的张量，每个内核对应于输入图像中的一个像素。每个内核都应应用于带噪图像中相应像素的邻居。这将生成网络的预测，然后可以计算预测和参考图像之间的损耗。

我试图创建一个自定义损失函数并处理其中的张量-我认为我有内核张量，损失函数内部也有噪点和参考图像，但是我不知道怎么做将内核应用于图像。这是我到目前为止的内容：

def kernelPredictMAV(self, y_true, y_pred):

    # y_pred.shape = (batch_size, image_width, image_height, k*k)
    # y_true.shape = (batch_size, image_width, image_height, 6)
    # The six comes from one reference RGB image and one noisy RGB image

    # Normalise the weights
    exp = tf.math.exp(y_pred)
    weight_sum = tf.reduce_sum(exp, axis=3, keepdims=True)
    weight_avg = tf.divide(exp, weight_sum)

    # Extract the two different images from the label
    reference_img = tf.slice(y_true, (0, 0, 0, 0), (batch_size, image_width, image_height, 3))
    noisy_img = tf.slice(y_true, (0, 0, 0, 3), (batch_size, image_width, image_height, 3))

    # Pad the noisy image by half the kernel size 
    kernel_radius = int(math.floor(k / 2.0))
    paddings = tf.constant([[0, 0], [kernel_radius, kernel_radius], [kernel_radius, kernel_radius], [0, 0]])
    noisy_img = tf.pad(noisy_img, paddings, mode="SYMMETRIC")

    # Reshape the k*k array into a kxk kernel
    y_pred = tf.reshape(y_pred, shape=[batch_size, image_width, image_height, k, k])

    ...

    # This is where I get stuck, I want something like
    y_pred = apply_kernels(y_pred, noisy_img)

    return tf.keras.losses.mean_absolute_value(y_pred, reference_img)

有什么方法可以在Keras中应用这样的内核吗？

如何在自定义Keras损失函数中应用具有不同内核的卷积

0 个答案: