基于模型输出的写损失函数,输出和损失之间没有梯度

时间:2019-04-06 09:03:22

标签: tensorflow

我正在尝试编写自己的损失函数来创建图像的对抗性示例,尤其是面孔。该损失函数的一个组成部分基于图像嵌入的余弦距离,而另一种嵌入是静态的。嵌入是Facenet https://github.com/davidsandberg/facenet模型的结果。我知道如何获得此嵌入。我面临的问题是,使用ADAM优化器将损失降到最低时,用于嵌入的损失分量与模型正在学习的新图像之间没有梯度。它会重新创建原始输入图像,但不会修改特定区域以更改生成的嵌入(嵌入和生成的图像之间没有梯度,只有生成的图像和起始图像之间没有渐变)。有没有一种方法可以基于Facenet模型的结果来创建损耗分量?以下代码基于https://github.com/carlini/nn_robust_attacks

    # the variable we're going to optimize over
    self.modifier = tf.Variable(np.zeros(shape, dtype=np.float32))

    # these are variables to be more efficient in sending data to tf
    self.timg = tf.Variable(np.zeros(shape), dtype=tf.float32)
    # self.tlab = tf.Variable(np.zeros((batch_size, num_labels)), dtype=tf.float32)
    self.const = tf.Variable(np.zeros(batch_size), dtype=tf.float32)

    # and here's what we use to assign them
    self.assign_timg = tf.placeholder(tf.float32, shape)
    # self.assign_tlab = tf.placeholder(tf.float32, (batch_size, num_labels))
    self.assign_const = tf.placeholder(tf.float32, [batch_size])

    self.boxmul = (boxmax - boxmin) / 2.
    self.boxplus = (boxmin + boxmax) / 2.

    self.nimg = self.modifier + self.timg
    self.newimg = tf.Variable(tf.tanh(self.nimg) * self.boxmul + self.boxplus)

    start_image = np.tanh(starting_image/255 ) * self.boxmul + self.boxplus

    mask = np.ones((160, 160, 3), dtype=np.float32)

    y_offset = msk.y_offset()
    x_offset = msk.x_offset()

    mask_indices = msk.return_mask()
    for idx, (j, i) in enumerate(mask_indices):
        mask[i + y_offset, j + x_offset, 0] = 0
        mask[i + y_offset, j + x_offset, 1] = 0
        mask[i + y_offset, j + x_offset, 2] = 0

    mask_tensor = tf.convert_to_tensor(mask)

    # get the difference between the starting image (masked) and the perturbation applied on top of the masked image
    self.img_variation = self.newimg[0] - start_image
    # self.img_variation = tf.Variable(diff_img)

    # get the difference in images (values bewteen pixels, masked areas are set to 0 so they don't count)
    self.img_loss = tf.reduce_sum(tf.square(tf.multiply(self.img_variation, mask_tensor)))

    self.embedding_output = tf.get_default_graph().get_tensor_by_name("embeddings:0")
    self.face_input = tf.get_default_graph().get_tensor_by_name("input:0")
    # get centroid
    centroid_embedding = auth.getCentroid(centroid_path)
    normalize_embedding = tf.nn.l2_normalize(self.embedding_output, 1)
    normalize_centroid = tf.nn.l2_normalize(centroid_embedding, 1)

    # get the cosine distance of the centroid to the calculated embedding
    self.cos_dist = tf.losses.cosine_distance(normalize_embedding, normalize_centroid, axis=1)

    # loss component for update region for perturbed image
    self.loss_update = tf.reduce_sum(abs(self.update_threshold + 0.01 - self.cos_dist))

    # total loss
    self.loss = self.loss_update + self.img_loss

    # Setup the adam optimizer and keep track of variables we're creating
    start_vars = set(x.name for x in tf.global_variables())
    optimizer = tf.train.AdamOptimizer(self.LEARNING_RATE)
    self.train = optimizer.minimize(self.loss, var_list=[self.newimg])
    end_vars = tf.global_variables()
    new_vars = [x for x in end_vars if x.name not in start_vars]

    # these are the variables to initialize when we run
    self.setup = []
    self.setup.append(self.timg.assign(self.assign_timg))
    # self.setup.append(self.tlab.assign(self.assign_tlab))
    self.setup.append(self.const.assign(self.assign_const))

    self.init = tf.variables_initializer(var_list=[self.newimg] + new_vars)
    self.init = tf.variables_initializer(var_list=[self.newimg])
    self.init = tf.global_variables_initializer()

然后是实际训练

    self.sess.run(self.setup, {self.assign_timg: [batch],
                                   # self.assign_tlab: batchlab,
                                   self.assign_const: CONST})

    prev = np.inf
    for iteration in range(self.MAX_ITERATIONS):
         print(iteration)
         # perform the attack

         img = np.round(np.arctanh((o_bestattack - self.boxplus) / self.boxmul * 0.999999) * 255)
         imgs = np.zeros((1, 160, 160, 3), np.float32)
         imgs[0] = img

         batch_size = tf.get_default_graph().get_tensor_by_name("batch_size:0")
         feed_dict = {self.face_input: imgs,
                         batch_size: 1,
                         phase_train_placeholder: False}

         _, l, scores, newimg = self.sess.run([self.train, self.loss,
                                                self.embedding_output, self.newimg], feed_dict=feed_dict)

0 个答案:

没有答案