Tensorflow：sparse_tensor_dense_matmul比常规matmul慢

Question

我有2种情况：

scenario 1:
op: sparse_tensor_dense_matmul
A: 1000x1000 sparsity = 90%
B: 1000x1000 sparsity = 0%

scenario 2:
op: matmul
A: 1000x1000 sparsity = 0%
B: 1000x1000 sparsity = 0%

我知道GPU不能很好地计算稀疏矩阵乘法，但是我当然希望它们能够至少执行非稀疏矩阵乘法运算。在我的代码中，sparse_tensor_dense_matmul的速度降低了10倍！

import tensorflow as tf
import numpy as np
import time
import itertools

rate = 0.1
N = 1000
itrs = 1000

num = int(rate * N * N)

combs = np.array(list(itertools.product(range(N), range(N))))
choices = range(len(combs))
_idxs = np.random.choice(a=choices, size=num, replace=False).tolist()
_idxs = combs[_idxs]
_idxs = _idxs.tolist()
_idxs = sorted(_idxs)

_vals = np.float32(np.random.rand(num))
_y = np.random.uniform(low=-1., high=1., size=(N, N))
_z = np.random.uniform(low=-1., high=1., size=(N, N))
################################################
x = tf.SparseTensor(indices=_idxs, values=_vals, dense_shape=(N, N))
y = tf.Variable(_y, dtype=tf.float32)
z = tf.Variable(_z, dtype=tf.float32)

sparse_dot = tf.sparse_tensor_dense_matmul(x, y)
dot = tf.matmul(z, y)
################################################
sess = tf.InteractiveSession()
tf.global_variables_initializer().run()
tf.local_variables_initializer().run()

start = time.time()
for i in range(itrs):
    [_sparse_dot] = sess.run([sparse_dot], feed_dict={})
total = time.time() - start
print (total)

start = time.time()
for i in range(itrs):
    [_dot] = sess.run([dot], feed_dict={})
total = time.time() - start
print (total)
################################################

25.357680797576904

2.7684502601623535