Question

我尝试构建一个可以通过数据识别并尝试查看LOSS功能的模型 loss = tf.reduce_mean（ - （y_ * tf.log（y）+（1- y _）* tf.log（1-y）））但截至目前，我只在预测中获得NAN并在LOSS函数中打印NAN

np_labels = np.array(labels)
np_labels = np_labels.reshape([np_labels.shape[0], 1])
features = 910
hidden_layer_nodes = 100
x = tf.placeholder(tf.float32, [None, features])
y_ = tf.placeholder(tf.float32, [None, 1])
W1 = tf.Variable(tf.truncated_normal([features,hidden_layer_nodes], stddev=0.1))
b1 = tf.Variable(tf.constant(0.1, shape=[hidden_layer_nodes]))
z1 = tf.add(tf.matmul(x,W1),b1)
a1 = tf.nn.relu(z1)
W2 = tf.Variable(tf.truncated_normal([hidden_layer_nodes,1], stddev=0.1))
b2 = tf.Variable(0.)
z2 = tf.matmul(a1,W2) + b2
y = 1 / (1.0 + tf.exp(-z2))

loss =tf.reduce_mean(-(y_ * tf.log(y)+(1- y_)* tf.log (1-y)))

update = tf.train.AdamOptimizer(0.01).minimize(loss)

sess = tf.Session()
sess.run(tf.global_variables_initializer())
for i in range(0,50):
        sess.run(update, feed_dict = {x:fvecs, y_:np_labels})
        print(sess.run(loss, feed_dict={x: fvecs, y_: np_labels}))

      #  sess.run(update, feed_dict = {x:data_x, y_:data_y})
     #   print(sess.run(loss, feed_dict={x: data_x, y_: data_y}))

print('prediction: ', y.eval(session=sess, feed_dict =  {x:[[493.9, 702.6, .....

我想打印损失

由于

Answer 1

这是不 TensorFlow-Issue。这是由于自己实现损失功能的非常糟糕的想法。

import tensorflow as tf

z2 = tf.random_normal([8, 10]) * 20
y_ = tf.random_uniform([8, 1], minval=0, maxval=10, dtype=tf.float32)

y = 1 / (1.0 + tf.exp(-z2))

loss = tf.reduce_mean(-(y_ * tf.log(y)+(1- y_)* tf.log (1-y)))

with tf.Session() as sess:
    print sess.run(loss)  # will always fail with high prob

将仅仅因为缺少log-sum-exp技巧而导致Inf，这会导致您的实现因数值不稳定而失败（民间传说示例会产生溢出）。只需运行此代码几次，即可获得NaN或Inf。

解决方案是：

将y = tf.sigmoid(-z2)替换为y = tf.identity(z2)以获取未转换的logits
将loss = ..替换为loss = tf.nn.sigmoid_cross_entropy_with_logits(...)以使用数值稳定方式

请参阅明确描述此问题的sigmoid_cross_entropy_with_logits文档。

纳损失函数张量流

1 个答案: