TensorFlow LinearRegressor不会收敛于简单回归

Question

我创建了一个虚拟回归数据集

from sklearn.datasets import make_regression
X, y = make_regression(n_features=1, noise=15)

如下所示：

然后使用Estimator API创建LinearRegressor

feat_cols = [tf.feature_column.numeric_column("X", shape=[1])]
estimator = tf.estimator.LinearRegressor(feature_columns=feat_cols)
input_function = tf.estimator.inputs.numpy_input_fn({"X": X}, y, batch_size=4, 
                                                    shuffle=True, num_epochs=None)
estimator.train(input_fn=input_function, steps=1000)

但由于某种原因，它并没有收敛

INFO:tensorflow:Calling model_fn.
INFO:tensorflow:Done calling model_fn.
INFO:tensorflow:Create CheckpointSaverHook.
INFO:tensorflow:Graph was finalized.
INFO:tensorflow:Running local_init_op.
INFO:tensorflow:Done running local_init_op.
INFO:tensorflow:Saving checkpoints for 1 into /tmp/tmpevbf55ce/model.ckpt.
INFO:tensorflow:loss = 32492.004, step = 1
INFO:tensorflow:global_step/sec: 356.786
INFO:tensorflow:loss = 28372.191, step = 101 (0.286 sec)
INFO:tensorflow:global_step/sec: 410.94
INFO:tensorflow:loss = 17810.11, step = 201 (0.241 sec)
INFO:tensorflow:global_step/sec: 473.588
INFO:tensorflow:loss = 23634.676, step = 301 (0.213 sec)
INFO:tensorflow:global_step/sec: 475.631
INFO:tensorflow:loss = 22925.46, step = 401 (0.210 sec)
INFO:tensorflow:global_step/sec: 381.111
INFO:tensorflow:loss = 35689.633, step = 501 (0.269 sec)
INFO:tensorflow:global_step/sec: 341.871
INFO:tensorflow:loss = 12368.4375, step = 601 (0.296 sec)
INFO:tensorflow:global_step/sec: 374.349
INFO:tensorflow:loss = 23559.09, step = 701 (0.257 sec)
INFO:tensorflow:global_step/sec: 379.525
INFO:tensorflow:loss = 8804.063, step = 801 (0.262 sec)
INFO:tensorflow:global_step/sec: 390.375
INFO:tensorflow:loss = 15016.509, step = 901 (0.256 sec)
INFO:tensorflow:Saving checkpoints for 1000 into /tmp/tmpevbf55ce/model.ckpt.
INFO:tensorflow:Loss for final step: 4406.3345.

我在SO上读到其他帖子，X和y的尺寸需要匹配，所以我也继续重塑它们：

input_function = tf.estimator.inputs.numpy_input_fn({"X": X)}, y.reshape(-1, 1), 
                                                    batch_size=4, shuffle=True, 
                                                    num_epochs=None)

但它仍然没有收敛。我可能遗漏了一些东西，是batch_size吗？或迭代次数？我已经尝试使用线性回归而不使用使用TensorFlow GradientDescentOptimizer的Estimator API并且它成功收敛。我不确定我到底错过了什么。

Answer 1

如果您在shuffle = false中设置tf.estimator.inputs.numpy_input_fn，如下所示，

input_function = tf.estimator.inputs.numpy_input_fn({"X": X}, Y, batch_size=4,
                                                    shuffle=False, num_epochs=None)

输出对我来说似乎很合理。

但是，我认为原因在于您的数据难以学习线性模型。当shuffle=False模型试图过度拟合而shuffle=True模型没有学到任何东西时。

<强> EDITED

您还可以按照以下步骤尝试 Gradientdescent 优化工具。

estimator = tf.estimator.LinearRegressor(feature_columns=feat_cols,optimizer=tf.train.GradientDescentOptimizer(learning_rate=1))

或者你可以继续使用相同的 FtrlOptimizer ，这是tf.estimator.LinearRegressor的默认优化器，学习率更高。

estimator = tf.estimator.LinearRegressor(feature_columns=feat_cols,optimizer=tf.train.FtrlOptimizer(learning_rate=1))

我认为两者都应该在shuffle = True时收敛。

但是，我不确定shuffle = True使用默认设置时未收敛的确切原因。

希望这有帮助。