张量流是否将(x * y)[0]优化为x[0] * y[0]？怎么确定？
我可以visualize原始计算图。如何可视化优化图？
索引编制似乎并不懒惰。
测试：
import tensorflow as tf
a = tf.ones(int(0.5e9)) # 0.5 GB
b = a[:, None] * a[None, :] # 0.25e9 GB
with tf.Session() as sess:
writer = tf.summary.FileWriter("output", sess.graph)
print(sess.run(b[0, 0]))
writer.close()
结果：
...
2019-06-04 14:42:11.788975:
W tensorflow/core/framework/op_kernel.cc:1546]
OP_REQUIRES failed at cwise_ops_common.cc:70 :
Resource exhausted:
OOM when allocating tensor with shape[500000000,500000000]
and type float on /job:localhost/replica:0/task:0/device:CPU:0
by allocator cpu
内存分配似乎并不懒惰。
测试：
import tensorflow as tf
a = tf.ones(int(20e9)) # 20 GB
b = tf.ones(int(20e9))
with tf.Session() as sess:
writer = tf.summary.FileWriter("output", sess.graph)
print(sess.run((a*b)[10]))
writer.close()
结果：
...
2019-06-04 14:37:25.304918:
W tensorflow/core/framework/cpu_allocator_impl.cc:81]
Allocation of 80000000000 exceeds 10% of system memory.
2019-06-04 14:37:25.304971:
W tensorflow/core/framework/cpu_allocator_impl.cc:81]
Allocation of 80000000000 exceeds 10% of system memory.
2019-06-04 14:37:25.304994:
F tensorflow/core/grappler/optimizers/constant_folding.cc:1171]
Check failed: value->FromProto(raw_val)
Aborted (core dumped)