Question

我想计算和评估关于输入的NN的雅可比行列式。我对构建雅可比人的时间并不十分关心，我更关心雅各比派的评价。

weights = {
    'w1': tf.Variable(tf.random_normal([num_input, num_hidden_1])),
    'w2': tf.Variable(tf.random_normal([num_hidden_1, num_hidden_2])),
    'w_final': tf.Variable(tf.random_normal([num_hidden_2, 1]))
}
biases = {
    'b1': tf.Variable(tf.random_normal([num_hidden_1])),
    'b2': tf.Variable(tf.random_normal([num_hidden_2])),
    'b_final': tf.Variable(tf.random_normal([num_hidden_2])),
}

def g(x):
    layer_1 = tf.nn.sigmoid(tf.add(tf.matmul(x, weights['w1']),
                                   biases['b1']))
    layer_2 = tf.nn.sigmoid(tf.add(tf.matmul(layer_1, weights['w2']),
                                   biases['b2']))

    final = tf.add(tf.matmul(layer_2, weights['w_final']),
                                   biases['b_final'])
    return final

现在计算jacobian

# https://github.com/tensorflow/tensorflow/issues/675
def jacobian(y_flat, x):
    n = y_flat.shape[0]

    loop_vars = [
        tf.constant(0, tf.int32),
        tf.TensorArray(tf.float32, size=n),
    ]

    _, jacobian = tf.while_loop(
        lambda j, _: j < n,
        lambda j, result: (j+1, result.write(j, tf.gradients(y_flat[j], x))),
        loop_vars)

    return jacobian.stack()

现在这需要很长时间：

i = 784
o = 1
n = 500

real_data = tf.placeholder(tf.float32, shape=[n, i])
g_app = g(real_data)
y=g_app
x=real_data

start = time.time()
j_4 = jacobian(y,x)
constructed = time.time()
print(str(int(constructed - start)) + "jacobian constructed")
loop = constructed
for z in range(0,10):
    j_out = sess.run(j_4, feed_dict={x:np.random.rand(n,i)})
    temp = time.time()
    print(str(int(temp - loop)) + " Seconds: " + str(j_out.shape))
    loop = temp

这是我的输出：

0 jacobian constructed
6 Seconds: (500, 1, 500, 784)
4 Seconds: (500, 1, 500, 784)
5 Seconds: (500, 1, 500, 784)
5 Seconds: (500, 1, 500, 784)
6 Seconds: (500, 1, 500, 784)
3 Seconds: (500, 1, 500, 784)
4 Seconds: (500, 1, 500, 784)
3 Seconds: (500, 1, 500, 784)
3 Seconds: (500, 1, 500, 784)
3 Seconds: (500, 1, 500, 784)

有没有办法加快速度？我没有看到为什么这么慢，理论上不是梯度下降的原因？

Answer 1

嗯，最简单的解决方案毕竟是最快的

def jacobian(y, x):
    with tf.name_scope("jacob"):
        grads = tf.stack([tf.gradients(yi, x)[0] for yi in tf.unstack(y, axis=1)],
                        axis=2)
        return grads

加快雅可比评估

1 个答案: