Question

我阅读了以下牛顿或lbfgs优化器的示例：

optimizer = ScipyOptimizerInterface(loss, options={'maxiter': 100})
with tf.Session() as session:
    optimizer.minimize(session)

但是我很困惑，因为它与梯度下降优化器的不同之处在于：

optimizer = tf.train.GradientDescentOptimizer(learning_rate).minimize(cost)
with tf.Session() as sess:
    sess.run(optimizer, feed_dict = {....})

我的问题是，可以按照下面的示例使用L-BFGS优化器吗？如果没有答案，如何在以下代码中使用L-BFGS优化器？

cost = compute_cost(ys, Z, W_soft, lambd)

train_soft = tf.train.GradientDescentOptimizer(0.5).minimize(cost)
correct_prediction = tf.equal(tf.argmax(Y_hat, 0), tf.argmax(ys, 0))
accuracy = tf.reduce_mean(tf.cast(correct_prediction, tf.float32))

init = tf.global_variables_initializer()

mini_batch_data, mini_batch_labels, mini_batch_nums = batch_split(train_X, train_Y, batch_size)

paras_soft = {}
with tf.Session() as sess:
    sess.run(init)
    for epoch in range(max_iter):
        train_soft_cost, _ = sess.run([cost, train_soft], feed_dict = {xs: train_X, ys: train_Y, keep_prob: 1})

Answer 1

ScipyOpimizerInterface是包装器，允许scipy.optimize.minimize在张量流Session中运行。

它接受method kwarg，您可以将docs中概述的任何方法传递给它。

optimizer = ScipyOptimizerInterface(loss, ..., method='L-BGFS-B')

您应该了解，方法的适用性取决于所使用的约束。请查阅文档以获取更多信息。

如何使用牛顿或L-BFGS作为优化程序？

1 个答案: