Question

我有一个使用keras的tensorflow后端的函数。在循环中，我向会话图添加操作，然后运行会话。问题在于，多次调用该函数后，图形似乎会大量增长。这导致在对函数进行4/5调用后，函数评估要长2倍。

这是功能：

def attack_fgsm(self, x, y, epsilon=1e-2):
    sess = K.get_session()
    nabla_x = np.zeros(x.shape)

    for (weak_classi, alpha) in zip(self.models, self.alphas):
        grads = K.gradients(K.categorical_crossentropy(y, weak_classi.model.output), weak_classi.model.input)[0]
        grads = sess.run(grads, feed_dict={weak_classi.model.input: x})
        nabla_x += alpha*grads

    x_adv = x + epsilon*np.sign(nabla_x)

    return x_adv

所以问题是如何优化此功能，以使图不会增长？

经过一些研究，看来我需要使用占位符来解决该问题。所以我想到了这个：

def attack_fgsm(self, x, y, epsilon=1e-2):
    sess = K.get_session()
    nabla_x = np.zeros(x.shape)
    y_ph = K.placeholder(y.shape)
    model_in = K.placeholder(x.shape, dtype="float")

    for (weak_classi, alpha) in zip(self.models, self.alphas):
        grads = K.gradients(K.categorical_crossentropy(y_ph, weak_classi.model.output), weak_classi.model.input)[0]
        grads = sess.run(grads, feed_dict={y_ph:y, model_in:x})
        nabla_x += alpha*grads

    x_adv = x + epsilon*np.sign(nabla_x)
    #K.clear_session()
    return x_adv

哪个指向：

Traceback (most recent call last):
  File "/home/simond/adversarialboosting/src/scripts/robustness_study.py", line 93, in <module>
    x_att_ada = adaboost.attack_fgsm(x_test, y_test, epsilon=eps)
  File "/home/simond/adversarialboosting/src/classes/AdvBoostM1.py", line 308, in attack_fgsm
    grads = sess.run(grads, feed_dict={y_ph:y, model_in:x})
  File "/home/simond/miniconda3/envs/keras/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 950, in run
    run_metadata_ptr)
  File "/home/simond/miniconda3/envs/keras/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1158, in _run
    self._graph, fetches, feed_dict_tensor, feed_handles=feed_handles)
  File "/home/simond/miniconda3/envs/keras/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 474, in __init__
    self._fetch_mapper = _FetchMapper.for_fetch(fetches)
  File "/home/simond/miniconda3/envs/keras/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 261, in for_fetch
    type(fetch)))
TypeError: Fetch argument None has invalid type <class 'NoneType'>

Answer 1

每次调用此函数时，问题都在运行以下代码行：

grads = K.gradients(K.categorical_crossentropy(y, weak_classi.model.output), weak_classi.model.input)[0]

这会将梯度的符号计算添加到图形中，并且不需要为每个weak_classi实例多次运行，因此您可以将其分为两部分。这部分应该只在初始化时运行一次，例如：

self.weak_classi_grads = []
for (weak_classi, alpha) in zip(self.models, self.alphas):
    grads = K.gradients(K.categorical_crossentropy(y_ph, weak_classi.model.output), weak_classi.model.input)[0]
self.weak_classi_grads.append(grads)

然后，您可以将评估函数重写为：

def attack_fgsm(self, x, y, epsilon=1e-2):
    sess = K.get_session()
    nabla_x = np.zeros(x.shape)

    for (weak_classi, alpha, grads) in zip(self.models, self.alphas, self.weak_classi_grads):
        grads = sess.run(grads, feed_dict={weak_classi.model.input: x})
        nabla_x += alpha*grads

    x_adv = x + epsilon*np.sign(nabla_x)

    return x_adv

通过这种方式，图仅对每个模型具有一个梯度计算实例，然后您只需要运行会话以评估具有不同输入的梯度。

避免张量流会话扩展

1 个答案: