Question

我正在使用Tensorflow Estimator API，但还没有弄清楚如何使用tf.contrib.opt.ScipyOptimizerInterface上的L-BFGS优化器。

似乎估算器API希望tf.train模块提供一些优化器，但是那里没有BFGS实现。 contrib中定义的唯一一个不遵循相同的接口。

更具体地说，在official tutorial中定义自定义估算器，它显示了如何使用AdagradOptimizer：

optimizer = tf.train.AdagradOptimizer(learning_rate=0.1)
train_op = optimizer.minimize(loss, global_step=tf.train.get_global_step())

不过，ScipyOptimizerInterface的API如下：

optimizer = ScipyOptimizerInterface(loss, options={'maxiter': 100})

with tf.Session() as session:
    optimizer.minimize(session)

举一个完整的例子：

from sklearn import datasets
import numpy as np


def _custom_model_fn(features, labels, mode, feature_columns):

    predictions = tf.feature_column.linear_model(features, feature_columns)
    predictions = tf.reshape(predictions, [-1])

    if mode == tf.estimator.ModeKeys.PREDICT:
        predictions = {'predictions': predictions}
        return tf.estimator.EstimatorSpec(mode, predictions=predictions)

    loss = tf.losses.mean_squared_error(labels=labels, predictions=predictions,
                                        reduction=tf.losses.Reduction.SUM_BY_NONZERO_WEIGHTS)

    if mode == tf.estimator.ModeKeys.EVAL:
        return tf.estimator.EstimatorSpec(mode, loss=loss, eval_metric_ops=metrics)

    # Create training op.
    assert mode == tf.estimator.ModeKeys.TRAIN

    # train_op = tf.contrib.opt.ScipyOptimizerInterface(loss, options={'maxiter': 10})

    optimizer = tf.train.FtrlOptimizer(learning_rate=1.)
    train_op = optimizer.minimize(loss, global_step=tf.train.get_global_step())
    return tf.estimator.EstimatorSpec(mode,
                                      predictions=predictions,
                                      loss=loss,
                                      train_op=train_op)

class MyRegressor(tf.estimator.Estimator):
    def __init__(self, feature_columns, model_dir=None, config=None):

        def _model_fn(features, labels, mode, config):
            return _custom_model_fn(features, labels, mode, feature_columns)

        super(MyRegressor, self).__init__(model_fn=_model_fn)

# Load the diabetes dataset
diabetes = datasets.load_diabetes()
diabetes_X = diabetes.data[:, np.newaxis, 2]
diabetes_y = diabetes.target

# Create the custom estimator and train it
feature_columns = [tf.feature_column.numeric_column('x')]
train_input_fn = tf.estimator.inputs.numpy_input_fn(
    x={'x': np.array(diabetes.data[:, 2])},
    y=np.array(diabetes.target),
    num_epochs=None,
    shuffle=True)
myregressor = MyRegressor(feature_columns)
myregressor.train(train_input_fn, steps=10000)

如果我取消注释使用ScipyOptimizer的行，显然会出现如下错误

TypeError: train_op must be Operation or Tensor, given: <tensorflow.contrib.opt.python.training.external_optimizer.ScipyOptimizerInterface object

有使用Scipy优化器的简单方法吗？

谢谢。

将L-BFGS优化器与Tensorflow估算器API结合使用

0 个答案: