Question

我目前正在尝试了解如何在Tensorflow的帮助下解决回归问题。不幸的是，只要我尝试为输入数据引入第二个维度，错误或丢失就会变得非常大。

我使用的数据集是自制的，非常简单。这些值都已排序，X2只是x1 + 1

中的每个值

X1 = [2.167,3.1,3.3,4.168,4.4,5.313,5.5,5.654,6.182,6.71,6.93,7.042,7.59,7.997,9.27,9.779,10.791]
X2 = [3.167,4.1,4.3,5.168,5.4,6.313,6.5,6.654,7.182,7.71,7.93,8.042,8.59,8.997,10.27,10.779,11.791]
y = [1.221,1.3,1.573,1.65,1.694,1.7,2.09,2.42,2.53,2.596,2.76,2.827,2.904,2.94,3.19,3.366,3.465]

我尝试使用线性回归来近似值：

numbers = pd.DataFrame({'x1': X1, 'x2':X2})

X_train, X_test, y_train, y_test = train_test_split(numbers,y,test_size=0.3,random_state=101)

X_data = tf.placeholder(shape=[None,2], dtype=tf.float32)
y_target = tf.placeholder(shape=[None], dtype=tf.float32)

w1 = tf.Variable(tf.random_normal(shape=[2,1])) 
b1 = tf.Variable(tf.random_normal(shape=[1]))

final_output = tf.add(tf.matmul(X_data, w1), b1)

loss = tf.reduce_sum(tf.square(final_output-y_target))

optimizer = tf.train.GradientDescentOptimizer(learning_rate=0.1)
train = optimizer.minimize(loss)

init = tf.global_variables_initializer()

steps = 5000

with tf.Session() as sess:

    sess.run(init)

    for i in range(steps):

        sess.run(train,feed_dict={X_data:X_train,y_target:y_train})

        # PRINT OUT A MESSAGE EVERY 100 STEPS
        if i%500 == 0:

            print('Currently on step {}'.format(i))

            training_cost = sess.run(loss, feed_dict={X_data:X_test,y_target:y_test})
            print("Training cost=", training_cost/5)

    training_cost = sess.run(loss, feed_dict={X_data:X_test,y_target:y_test})
    print("Training cost=", training_cost)

这给了我输出

Currently on step 0
Training cost= 12376958566.4
Currently on step 500
Training cost= nan
Currently on step 1000
Training cost= nan
Currently on step 1500
Training cost= nan
Currently on step 2000
Training cost= nan
Currently on step 2500
Training cost= nan
Currently on step 3000
Training cost= nan
Currently on step 3500
Training cost= nan
Currently on step 4000
Training cost= nan
Currently on step 4500
Training cost= nan
Training cost= nan

我使用Adagrad优化器获得了一些更好的结果，这给我一个5的错误，但我仍然认为应该有更多可能。

这可能是添加隐藏图层的选项吗？我之前尝试过这个，但是当我在层中使用relu作为激活函数并且在输出层上只使用f(x)=x时，我收到了类似的高纳米错误。

Answer 1

有两个问题。首先，您使用tf.reduce_sum代替tf.reduce_mean，因此您拥有的数据越多，您的损失就越大。这会向GradientDescentOptimizer发送一个巨大的信号，因此学到的重量会产生巨大的跳跃，并且您的模型会发散。

第二个问题是你的学习率仍然太高;正如user2015762建议的那样，当你有越来越多的损失时，这通常是个问题。我使用learning_rate=0.001代码获得了良好的效果。

回归练习中的巨大错误

1 个答案: