Question

我是DL和Keras的新手，目前我正在尝试在Keras中实现基于sobel-filter的自定义丢失功能。

这个想法是计算索贝尔滤波预测和索贝尔滤波地面实况图像的均方损失。

到目前为止，我的自定义丢失函数如下所示：

from scipy import ndimage

def mse_sobel(y_true, y_pred):

    for i in range (0, y_true.shape[0]):
        dx_true = ndimage.sobel(y_true[i,:,:,:], 1)
        dy_true = ndimage.sobel(y_true[i,:,:,:], 2)
        mag_true[i,:,:,:] = np.hypot(dx_true, dy_true)
        mag_true[i,:,:,:] *= 1.0 / np.max(mag_true[i,:,:,:])

        dx_pred = ndimage.sobel(y_pred[i,:,:,:], 1)
        dy_pred = ndimage.sobel(y_pred[i,:,:,:], 2)
        mag_pred[i,:,:,:] = np.hypot(dx_pred, dy_pred)
        mag_pred[i,:,:,:] *= 1.0 / np.max(mag_pred[i,:,:,:])

    return(K.mean(K.square(mag_pred - mag_true), axis=-1))

使用此丢失功能会导致此错误：

in mse_sobel
for i in range (0, y_true.shape[0]):
TypeError: __index__ returned non-int (type NoneType)

使用我发现的调试器，y_true.shape只返回None - 很好。但是，如果我将y_true.shape替换为例如1，使其看起来像for i in range (0,1):，则会发生另一个错误：

in sobel
axis = _ni_support._check_axis(axis, input.ndim)

in _check_axis
raise ValueError('invalid axis')
ValueError: invalid axis

在这里，我不确定为什么轴似乎无效？

任何人都可以帮我弄清楚如何实现这种损失功能吗？非常感谢你的帮助！

Answer 1

必须使用keras后端或tensorflow / theano / cntk函数进行张量运算。这是保持反向传播的唯一方法。使用numpy，scipy等会破坏图表。

让我们导入keras后端：

import keras.backend as K

定义过滤器：

#this contains both X and Y sobel filters in the format (3,3,1,2)
#size is 3 x 3, it considers 1 input channel and has two output channels: X and Y
sobelFilter = K.variable([[[[1.,  1.]], [[0.,  2.]],[[-1.,  1.]]],
                      [[[2.,  0.]], [[0.,  0.]],[[-2.,  0.]]],
                      [[[1., -1.]], [[0., -2.]],[[-1., -1.]]]])

这里是一个为每个输入通道重复过滤器的功能，以防您的图像是RGB或有多个通道。这将只复制每个输入通道的sobel过滤器：(3,3,inputChannels, 2)：

def expandedSobel(inputTensor):

    #this considers data_format = 'channels_last'
    inputChannels = K.reshape(K.ones_like(inputTensor[0,0,0,:]),(1,1,-1,1))
    #if you're using 'channels_first', use inputTensor[0,:,0,0] above

    return sobelFilter * inputChannels

这就是损失函数：

def sobelLoss(yTrue,yPred):

    #get the sobel filter repeated for each input channel
    filt = expandedSobel(yTrue)

    #calculate the sobel filters for yTrue and yPred
    #this generates twice the number of input channels 
    #a X and Y channel for each input channel
    sobelTrue = K.depthwise_conv2d(yTrue,filt)
    sobelPred = K.depthwise_conv2d(yPred,filt)

    #now you just apply the mse:
    return K.mean(K.square(sobelTrue - sobelPred))

在模型中应用此损失：

model.compile(loss=sobelLoss, optimizer = ....)

我的经验表明，计算统一的索贝尔滤波器sqrt(X² + Y²)会带来可怕的结果，结果图像听起来像棋盘。但如果你确实想要它：

def squareSobelLoss(yTrue,yPred):

    #same beginning as the other loss
    filt = expandedSobel(yTrue)
    squareSobelTrue = K.square(K.depthwise_conv2d(yTrue,filt))
    squareSobelPred = K.square(K.depthwise_conv2d(yPred,filt))

    #here, since we've got 6 output channels (for an RGB image)
    #let's reorganize in order to easily sum X² and Y²: change (h,w,6) to (h,w,3,2)
    #caution: this method of reshaping only works in tensorflow
    #if you do need this in other backends, let me know
    newShape = K.shape(squareSobelTrue)
    newShape = K.concatenate([newShape[:-1],
                              newShape[-1:]//2,
                              K.variable([2],dtype='int32')])

    #sum the last axis (the one that is 2 above, representing X² and Y²)                      
    squareSobelTrue = K.sum(K.reshape(squareSobelTrue,newShape),axis=-1)
    squareSobelPred = K.sum(K.reshape(squareSobelPred,newShape),axis=-1)

    #since both previous values are already squared, maybe we shouldn't square them again? 
    #but you can apply the K.sqrt() in both, and then make the difference, 
    #and then another square, it's up to you...    
    return K.mean(K.abs(squareSobelTrue - squareSobelPred))

如何使用Keras

1 个答案: