Question

我正在尝试使用GradientDescentOptimizer进行线性回归，但是得到的结果是我的错误实际上增长很快，然后溢出。我在做什么错了？

这是每次迭代中我的错误的示例结果：

2163732.5
1274220300000000.0
7.274338e+23
4.141076e+32
inf
inf
...

这是我的代码

import os 
import pandas as pd
import tensorflow as tf
from sklearn.model_selection import train_test_split

os.environ['TF_CPP_MIN_LOG_LEVEL'] = '2'
tf.logging.set_verbosity(tf.logging.ERROR)

data = pd.read_csv('test.csv').values
x_vals = data[:,1:]
y_vals = data[:,0]
n_dim = x_vals.shape[1]

W = tf.Variable(tf.ones([1, n_dim]))
b = tf.Variable(0.5, dtype=tf.float32)

X = tf.placeholder(tf.float32)
Y = tf.placeholder(tf.float32)

prediction = tf.reduce_sum(W * X) + b
error = Y - prediction
loss = tf.reduce_mean(tf.square(error))

optimizer = tf.train.GradientDescentOptimizer(0.01).minimize(loss)

with tf.Session() as sess:

    init = tf.initializers.global_variables()
    sess.run(init)

    for i in range (0, 100):

        x_train, x_test, y_train, y_test = train_test_split(x_vals, y_vals, test_size=100, train_size=100)

        _, loss_result = sess.run([optimizer, loss], {X: x_train, Y: y_train})
        print(loss_result)

我使用公式y = (0.5 * x_1) + (3 * x_2)生成了数据，因此它应该是完全线性的（忽略舍入错误）：看起来像这样：

y,x_1,x_2
28,9,8
24,6,7
31,9,9
34,8,10
24,12,6
...

Here's my full data

Answer 1

您的梯度超出了最小值，因此发生了爆炸。您应该尝试增加纪元数并将学习率降低到1e-5或更低一些，例如1e-7，1e-8。对于值epoch = 100000和学习率= 0.0000003，它不会超调。

GradientDescentOptimizer错误持续增长

1 个答案: