我正在尝试创建类似此项目https://github.com/tensorflow/lucid/tree/master/lucid/optvis的过滤器可视化效果
我设法使一些幼稚的代码可以一次生成一张图像
@objc sampleFunction() -> Int{
//do some stuff that's important when the button is pressed
return 5
}
但是它非常慢,因为它一次只能生成一张图像。感觉像是在浪费GPU周期。
然后,我想到了一次尝试以与图层中滤镜数量相同的大小同时生成一批图像的方法。
只有我不知道如何针对像轴上的张量的每个切片生成梯度
这是我所了解的,我只收到了None类型的列表
import time
import os
import multiprocessing as mp
import numpy as np
import tensorflow as tf
from tensorflow.keras.applications.vgg16 import preprocess_input
import matplotlib.pyplot as plt
from PIL import Image
os.environ["CUDA_VISIBLE_DEVICES"]="1"
# Layer name to inspect
layer_name = 'block5_conv2'
temp = tf.zeros([4, 32, 32, 3]) # Or tf.zeros
preprocess_input(temp)
epochs = 1500
step_size = .05
# filter_index = 1
# Create a connection between the input and the target layer
model = tf.keras.models.load_model('base_model_cifar10_vgg16.h5')
# Initiate random noise
input_img_data = np.random.random((1, 224, 224, 3))
input_img_data = (input_img_data - 0.5) * 20 + 128.
# Cast random noise from np.float64 to tf.float32 Variable
targets = []
for i, layer in enumerate(model.layers):
if 'conv' in layer.name:
targets.append(layer.name)
targets
%%time
for layer_name in targets:
print(f'generating dataset {layer_name}')
submodel = tf.keras.models.Model([model.inputs[0]],
[model.get_layer(layer_name).output])
for filter_index in range(submodel.output.get_shape()[-1]):
input_img_data = np.random.random((1, 224, 224, 3))
input_img_data = (input_img_data - 0.5) * 20 + 128.
input_img_data = tf.Variable(tf.cast(input_img_data, tf.float32))
for _ in range(epochs):
with tf.GradientTape() as tape:
outputs = submodel(preprocess_input(input_img_data))
loss_value = tf.reduce_mean(outputs[:, :, :, filter_index])
grads = tape.gradient(loss_value, input_img_data)
normalized_grads = grads / (tf.sqrt(tf.reduce_mean(tf.square(grads))) + 1e-5)
input_img_data.assign_add(normalized_grads * step_size)
image = input_img_data.numpy()[0,:,:,:].astype(np.uint8)
p = mp.Process(target=Image.fromarray(image).save(fp=f'./synth_sets/vgg16_cifar10/{layer_name}_{filter_index}.jpg'))
p.start()
有人知道我如何计算每个图像切片的单个梯度吗?