Question

我正在使用Keras创建梯度类激活图以可视化我的预测，但是对于我经过的某些图像，对于特定特征图通道上的梯度平均强度，它返回零。我正在使用DenseNet121在两个类之间进行分类。
我遵循了Francois Chollet在其《用Python进行深度学习》一书中使用Grad Cams编写的指南，并且我对VGG16模型也存在完全相同的问题，因此我假设这与我的模型无关选择。

我的模型如下：

K.set_image_dim_ordering('tf')

dnet = DenseNet121(include_top=True, weights='imagenet', input_shape=(img_size, img_size, 3))
dnet.trainable=True
x = dnet.output
x = layers.Dense(100, activation='relu')(x)
x = layers.Dropout(0.4)(x)
x = layers.Dense(2, activation='sigmoid')(x)
model = Model(input=dnet.input, output=x)

optimizer = Adam(lr=0.0001, beta_1=0.9, beta_2=0.999, decay=0.08) 
class_weight = {0:cwr, 1:1}

model.compile(loss='binary_crossentropy', optimizer=optimizer, metrics=['accuracy'])
model_hist = model.fit_generator(train, steps_per_epoch = 60, epochs = 2, class_weight = class_weight)

Grad CAM和相关热图的代码：

img = load_img(load_path, target_size=(img_size, img_size))
img_tensor = img_to_array(img)
img_tensor = np.expand_dims(img_tensor, axis=0)
x = preprocess_input(img_tensor)
preds = model.predict(x)
t = np.argmax(preds[0])
img = cv2.imread(str(load_path))

llayer = model.output[:,t]
last_conv_layer = model.get_layer(conv)
grads = K.gradients(llayer, last_conv_layer.output)[0]
pooled_grads = K.mean(grads, axis=(0, 1, 2))
iterate = K.function([model.input],[pooled_grads, last_conv_layer.output[0]])
pooled_grads_value, conv_layer_output_value = iterate([img_tensor])
for i in range(len(pooled_grads_value)):
    conv_layer_output_value[:, :, i] *= pooled_grads_value[i]


heatmap = np.mean(conv_layer_output_value, axis=-1)
heatmap = np.maximum(heatmap, 0)
heatmap /= (np.max(heatmap))
heatmap = cv2.resize(heatmap, (img.shape[1], img.shape[0]))
heatmap = np.uint8(255 * heatmap)
heatmap = cv2.applyColorMap(heatmap, cv2.COLORMAP_JET)
cv2.imwrite('colormap.jpg', heatmap)
colormap = cv2.imread('colormap.jpg')

我尽可能地将问题本地化到pooled_grads_value。打印此值将给出

[0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]

在尝试绘制关联的热图时也返回以下错误

/opt/conda/lib/python3.6/site-packages/ipykernel_launcher.py:85: RuntimeWarning: invalid value encountered in true_divide

正常工作的图像示例将返回pooled_grads_value作为类似的

[ 3.52818736e-16 -2.74286623e-17 -1.04039105e-16  2.26564966e-15
  3.80025990e-16 -4.65492462e-16 -3.13070048e-16 -3.99670802e-16
 -4.33913274e-16 -1.11373781e-16 -2.18853726e-16 -3.50514463e-16
  1.03881816e-17 -6.30468010e-16 -5.57306545e-16 -1.23719115e-16
 -3.93387115e-17  7.59074981e-16 -5.14396333e-16 -1.02742529e-16
 -2.16168923e-16  1.81140590e-16 -6.42374594e-16  3.01582507e-17
 -5.55568844e-17 -3.05854862e-16  3.26082836e-17  2.35498082e-16
  7.86424100e-18  6.45698563e-16 -1.54681729e-16  1.11217808e-16]

类激活图（CAM）的合并梯度问题

0 个答案: