Question

使用tensorflow（1.4版）时，在调试代码时遇到了一些问题。 Tensorflow有一些严重的精度错误故障。我在这里展示了一个例子。有没有办法在tensorflow中处理这类问题？

>>> a = [18.00146484]
>>> b= [[18.00146484]]
>>> c = [[[18.00146484]]]
>>> d = [[18.00146484],[12.83231735]]
>>> e = [[[18.00146484],[12.83231735]]]
>>> q = tf.nn.sigmoid()
KeyboardInterrupt
>>> q = tf.nn.sigmoid(a)
>>> w = tf.nn.sigmoid(b)
>>> e = tf.nn.sigmoid(c)
>>> r = tf.nn.sigmoid(d)
>>> z = [[[18.00146484],[12.83231735]]]
>>> t = tf.nn.sigmoid(z)
>>> init = tf.global_variables_initializer()
>>> sess = tf.Session()
2017-12-12 15:41:53.766287: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX
>>> qq,ww,ee,rr,tt = sess.run([q,w,e,r,t])
>>> qq,ww,ee,rr,tt
(array([ 1.], dtype=float32), array([[ 1.]], dtype=float32), array([[[ 1.]]], dtype=float32), array([[ 1.        ],
       [ 0.99999738]], dtype=float32), array([[[ 1.        ],
        [ 0.99999738]]], dtype=float32))
>>> qq
array([ 1.], dtype=float32)
>>> ww
array([[ 1.]], dtype=float32)
>>> ee
array([[[ 1.]]], dtype=float32)
>>> rrr
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
NameError: name 'rrr' is not defined
>>> rr
array([[ 1.        ],
       [ 0.99999738]], dtype=float32)
>>> tt
array([[[ 1.        ],
        [ 0.99999738]]], dtype=float32)
>>> from math import exp
>>> a = 18.00146484
>>> b = (1/(1+exp(-a)))
>>> b
0.9999999847923136
>>>

Answer 1

首先，看看你有多少试验，也许你觉得张量流对于相同的初始值给出了不同的结果。似乎并非如此。 sigmoid(18.00146484)始终为1，sigmoid(12.83231735)始终为0.99999738。

也许您认为sigmoid(18.00146484)与1相距足够远，不应该舍入为1，但事实并非如此。 sigmoid(18.00146484) = 0.99999998479231364...（https://www.wolframalpha.com/input/?i=sigmoid(18.00146484)），对于float32 precision，此数字太接近1。您确实需要在此级别的精度中使用double（tf.float64张量流）。这是一个简单的C ++程序，它为float和double显示最接近的可表示数字1。

#include <limits>
#include <cmath>
#include <iostream>

int main() {
  typedef std::numeric_limits< double > dbl;
  typedef std::numeric_limits< double > flt;

  double sigmoid = 0.99999998479231364;

  float x = 1.0;
  double firstSmallerThanX = std::nextafter(x, 0.0f);
  std::cout.precision(flt::max_digits10);
  std::cout << std::fixed << firstSmallerThanX << std::endl;
  std::cout << "Sigmoid is bigger than firstSmallerThanX: " << (sigmoid > firstSmallerThanX) << std::endl;

  double y = 1.0;
  double firstSmallerThanY = std::nextafter(y, 0.0);
  std::cout.precision(dbl::max_digits10);
  std::cout << std::fixed << firstSmallerThanY << std::endl;
  std::cout << "Sigmoid is smaller than firstSmallerThanY: " << (sigmoid < firstSmallerThanY) << std::endl;

  return 0;
}

这个版画：

0.99999994039535522
Sigmoid is bigger than firstSmallerThanX: 1
0.99999999999999989
Sigmoid is smaller than firstSmallerThanY: 1

张量流中浮点精度

1 个答案: