我们如何使用TensorFlow while循环（tf.while_loop）实现基于梯度的神经网络可视化？

Question

为了知道卷积神经网络的卷积层中某个过滤器对什么敏感，可以应用基于梯度的过滤器可视化。

这个想法是将随机图像馈送到网络中，然后找到使滤波器特征图激活最大化的梯度。将这些渐变添加到图像并进行迭代。

让：

conv_node be a convolutional layer in a neural network

filter_index be the index of the filter we want to visualize

建议的方法可以在这里找到：https://github.com/penny4860/cnn-visualizer/blob/master/src/utils.py

我们可以使用以下伪代码在TensorFlow中实现建议的方法：

loss = average(get_feature_map(conv_node, filter_index))
gradients = gradients(loss, input_image)
gradients = normalize(gradients)

使用TensorFlow会话：

gradients_values = session.run(gradients)
random_input_image + = gradients_values

我的问题是：

如何使用TensorFlow的while_loop API实现相同的过程？

Answer 1

自定义回调是在训练，评估或推理期间自定义张量流模型行为的强大工具。

您可以利用callbacks完成任务。下面是一个示例，其中我们在每个历元之后可视化model.layer[4]的核，并在每个历元之后捕获渐变。同样，您可以使用内置方法创建自己的自定义函数。您可以在这里找到更多相关信息-https://www.tensorflow.org/guide/keras/custom_callback

注意：我正在使用tensorflow 1.15.0

# Importing dependency
%tensorflow_version 1.x
import tensorflow as tf
from tensorflow.keras import backend as K
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Dense, Activation, Dropout, Flatten, Conv2D, MaxPooling2D, BatchNormalization
import numpy as np
from matplotlib import pyplot

np.random.seed(1000)

# Get Data
import tflearn.datasets.oxflower17 as oxflower17
x, y = oxflower17.load_data(one_hot=True)

# Create a sequential model
model = Sequential()

# 1st Convolutional Layer
model.add(Conv2D(filters=5, input_shape=(224,224,3), kernel_size=(3,3), strides=(1,1), padding='same'))
model.add(Activation('relu'))
# Pooling 
model.add(MaxPooling2D(pool_size=(2,2), strides=(2,2), padding='valid'))
# Batch Normalisation before passing it to the next layer
model.add(BatchNormalization())

# 2nd Convolutional Layer
model.add(Conv2D(filters=10, kernel_size=(3,3), strides=(1,1), padding='same'))
model.add(Activation('relu'))
# Pooling
model.add(MaxPooling2D(pool_size=(2,2), strides=(2,2), padding='valid'))
# Batch Normalisation
model.add(BatchNormalization())

# 3rd Convolutional Layer
model.add(Conv2D(filters=5, kernel_size=(3,3), strides=(1,1), padding='same'))
model.add(Activation('relu'))
# Pooling
model.add(MaxPooling2D(pool_size=(2,2), strides=(2,2), padding='valid'))
# Batch Normalisation
model.add(BatchNormalization())

# Passing it to a dense layer
model.add(Flatten())

# 1st Dense Layer
model.add(Dense(5, input_shape=(224*224*3,)))
model.add(Activation('relu'))

# Output Layer
model.add(Dense(17))
model.add(Activation('softmax'))

model.summary()

# Compile 
model.compile(loss='categorical_crossentropy', optimizer='adam', metrics=['accuracy'])

epoch_gradient = []
epoch_count = 0

def get_gradient_func(model):
    grads = K.gradients(model.total_loss, model.trainable_weights)
    inputs = model._feed_inputs + model._feed_targets + model._feed_sample_weights
    func = K.function(inputs, grads)
    return func

# Define the Required Callback Function
class GradientCalcCallback(tf.keras.callbacks.Callback):
  def on_epoch_end(self, epoch, logs=None):
      print("\n","Calculating Gradient for Epoch ",(epoch+1))
      get_gradient = get_gradient_func(model)
      grads = get_gradient([x, y, np.ones(len(y))])
      epoch_gradient.append(grads)

      # Visualize the Kernels for Layer 5 of the Model
      print("\n","Visualizing the kernels for Layer 5 of the Model for Epoch ",(epoch+1))  
      # retrieve weights from the second hidden layer
      filters, biases = model.layers[4].get_weights()
      # normalize filter values to 0-1 so we can visualize them
      f_min, f_max = filters.min(), filters.max()
      filters = (filters - f_min) / (f_max - f_min)
      # plot all the filters
      # n_filters = outgoing filters
      n_filters, ix = 10, 1 
      for i in range(n_filters):
        # get the filter
        f = filters[:, :, :, i]
        # Range of incoming filters
        for j in range(5):
            # specify subplot and turn of axis
            ax = pyplot.subplot(10, 5, ix)
            ax.set_xticks([])
            ax.set_yticks([])
            # plot filter channel in grayscale
            pyplot.imshow(f[:, :, j], cmap='gray')
            ix += 1
      # show the figure
      pyplot.show()

epoch = 4

model.fit(x, y, batch_size=64, epochs= epoch, verbose=1, validation_split=0.2, shuffle=True, callbacks=[GradientCalcCallback()])

# Convert to a array
gradient = np.asarray(epoch_gradient)
print("Shape of the Captured Gradient Array :",gradient.Shape)

输出-

Model: "sequential_29"
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
conv2d_87 (Conv2D)           (None, 224, 224, 5)       140       
_________________________________________________________________
activation_145 (Activation)  (None, 224, 224, 5)       0         
_________________________________________________________________
max_pooling2d_24 (MaxPooling (None, 112, 112, 5)       0         
_________________________________________________________________
batch_normalization_24 (Batc (None, 112, 112, 5)       20        
_________________________________________________________________
conv2d_88 (Conv2D)           (None, 112, 112, 10)      460       
_________________________________________________________________
activation_146 (Activation)  (None, 112, 112, 10)      0         
_________________________________________________________________
max_pooling2d_25 (MaxPooling (None, 56, 56, 10)        0         
_________________________________________________________________
batch_normalization_25 (Batc (None, 56, 56, 10)        40        
_________________________________________________________________
conv2d_89 (Conv2D)           (None, 56, 56, 5)         455       
_________________________________________________________________
activation_147 (Activation)  (None, 56, 56, 5)         0         
_________________________________________________________________
max_pooling2d_26 (MaxPooling (None, 28, 28, 5)         0         
_________________________________________________________________
batch_normalization_26 (Batc (None, 28, 28, 5)         20        
_________________________________________________________________
flatten_29 (Flatten)         (None, 3920)              0         
_________________________________________________________________
dense_58 (Dense)             (None, 5)                 19605     
_________________________________________________________________
activation_148 (Activation)  (None, 5)                 0         
_________________________________________________________________
dense_59 (Dense)             (None, 17)                102       
_________________________________________________________________
activation_149 (Activation)  (None, 17)                0         
=================================================================
Total params: 20,842
Trainable params: 20,802
Non-trainable params: 40
_________________________________________________________________
Train on 1088 samples, validate on 272 samples
Epoch 1/4
 960/1088 [=========================>....] - ETA: 0s - loss: 2.8102 - acc: 0.1094
 Calculating Gradient for Epoch  1

 Visualizing the kernels for Layer 5 of the Model for Epoch  1

1088/1088 [==============================] - 9s 8ms/sample - loss: 2.7977 - acc: 0.1121 - val_loss: 2.8206 - val_acc: 0.1250
Epoch 2/4
 960/1088 [=========================>....] - ETA: 0s - loss: 2.5060 - acc: 0.1979
 Calculating Gradient for Epoch  2

 Visualizing the kernels for Layer 5 of the Model for Epoch  2

1088/1088 [==============================] - 6s 5ms/sample - loss: 2.5227 - acc: 0.1921 - val_loss: 2.8027 - val_acc: 0.1140
Epoch 3/4
 960/1088 [=========================>....] - ETA: 0s - loss: 2.3459 - acc: 0.2583
 Calculating Gradient for Epoch  3

 Visualizing the kernels for Layer 5 of the Model for Epoch  3

1088/1088 [==============================] - 5s 5ms/sample - loss: 2.3493 - acc: 0.2592 - val_loss: 2.7985 - val_acc: 0.0956
Epoch 4/4
 960/1088 [=========================>....] - ETA: 0s - loss: 2.1954 - acc: 0.3063
 Calculating Gradient for Epoch  4

 Visualizing the kernels for Layer 5 of the Model for Epoch  4

1088/1088 [==============================] - 6s 5ms/sample - loss: 2.1978 - acc: 0.3006 - val_loss: 2.8202 - val_acc: 0.0551
Shape of the Captured Gradient Array : (4, 16)