Question

我有兴趣在TensorFlow中对Kronecker Recurrent Units实施this paper。

这涉及Kronecker产品的计算。 TensorFlow没有Kronecker产品的操作。我正在寻找一种有效而强大的计算方法。

这是否存在，还是需要手动定义TensorFlow操作？

Answer 1

如果您将阅读conv2d_transpose的数学定义并查看Kronecker product计算的内容，您会看到conv2d_tranpose的适当大小的stides（第二个矩阵的宽度，高度）），它做同样的事情。

此外，您甚至可以使用conv2d_transpose开箱即用Kronecker的产品。

以下是您从wiki计算Kronecker的矩阵产品的示例。

import tensorflow as tf
a = [[1, 2], [3, 4]]
b = [[0, 5], [6, 7]]

i, k, s = len(a), len(b), len(b)
o = s * (i - 1) + k

a_tf  = tf.reshape(tf.constant(a, dtype=tf.float32), [1, i, i, 1])
b_tf = tf.reshape(tf.constant(b, dtype=tf.float32), [k, k, 1, 1])

res = tf.squeeze(tf.nn.conv2d_transpose(a_tf, b_tf, (1, o, o, 1), [1, s, s, 1], "VALID"))

with tf.Session() as sess:
    print sess.run(res)

请注意，对于非方形矩阵，您需要在行中计算更多维度：

i, k, s = len(a), len(b), len(b)
o = s * (i - 1) + k

并正确使用它们作为你的步幅/输出参数。

Answer 2

这是我用于此的实用程序。有关用法示例，请参阅kronecker_test

def fix_shape(tf_shape):
  return tuple(int(dim) for dim in tf_shape)

def concat_blocks(blocks, validate_dims=True):
  """Takes 2d grid of blocks representing matrices and concatenates to single
  matrix (aka ArrayFlatten)"""

  if validate_dims:
    col_dims = np.array([[int(b.shape[1]) for b in row] for row in blocks])
    col_sums = col_dims.sum(1)
    assert (col_sums[0] == col_sums).all()
    row_dims = np.array([[int(b.shape[0]) for b in row] for row in blocks])
    row_sums = row_dims.sum(0)
    assert (row_sums[0] == row_sums).all()

  block_rows = [tf.concat(row, axis=1) for row in blocks]
  return tf.concat(block_rows, axis=0)

def chunks(l, n):
  """Yield successive n-sized chunks from l."""
  for i in range(0, len(l), n):
    yield l[i:i + n]

from tensorflow.python.framework import ops
original_shape_func = ops.set_shapes_for_outputs
def disable_shape_inference():
  ops.set_shapes_for_outputs = lambda _: _

def enable_shape_inference():
  ops.set_shapes_for_outputs = original_shape_func

def kronecker(A, B, do_shape_inference=True):
  """Kronecker product of A,B.
  turn_off_shape_inference: if True, makes 10x10 kron go 2.4 sec -> 0.9 sec
  """

  Arows, Acols = fix_shape(A.shape)
  Brows, Bcols = fix_shape(B.shape)
  Crows, Ccols = Arows*Brows, Acols*Bcols

  temp = tf.reshape(A, [-1, 1, 1])*tf.expand_dims(B, 0)
  Bshape = tf.constant((Brows, Bcols))

  # turn off shape inference
  if not do_shape_inference:
    disable_shape_inference()

  # [1, n, m] => [n, m]
  slices = [tf.reshape(s, Bshape) for s in tf.split(temp, Crows)]

  #  import pdb; pdb.set_trace()
  grid = list(chunks(slices, Acols))
  assert len(grid) == Arows
  result = concat_blocks(grid, validate_dims=do_shape_inference)

  if not do_shape_inference:
    enable_shape_inference()
    result.set_shape((Arows*Brows, Acols*Bcols))

  return result

def kronecker_test():
  A0 = [[1,2],[3,4]]
  B0 = [[6,7],[8,9]]
  A = tf.constant(A0)
  B = tf.constant(B0)
  C = kronecker(A, B)
  sess = tf.Session()
  C0 = sess.run(C)
  Ct = [[6, 7, 12, 14], [8, 9, 16, 18], [18, 21, 24, 28], [24, 27, 32, 36]]
  Cnp = np.kron(A0, B0)
  check_equal(C0, Ct)
  check_equal(C0, Cnp)

Answer 3

TensorFlow 1.7+在tf.contrib.kfac.utils.kronecker_product中提供了kronecker_product函数：

a = tf.eye(3)
b = tf.constant([[1., 2.], [3., 4.]])
kron = tf.contrib.kfac.utils.kronecker_product(a, b)

tf.Session().run(kron)

输出：

array([[1., 2., 0., 0., 0., 0.],
       [3., 4., 0., 0., 0., 0.],
       [0., 0., 1., 2., 0., 0.],
       [0., 0., 3., 4., 0., 0.],
       [0., 0., 0., 0., 1., 2.],
       [0., 0., 0., 0., 3., 4.]], dtype=float32)

Answer 4

尝试以下解决方案，看看它是否适合您：

def tf_kron(a,b):
    a_shape = [a.shape[0].value,a.shape[1].value]
    b_shape = [b.shape[0].value,b.shape[1].value]
    return tf.reshape(tf.reshape(a,[a_shape[0],1,a_shape[1],1])*tf.reshape(b,[1,b_shape[0],1,b_shape[1]]),[a_shape[0]*b_shape[0],a_shape[1]*b_shape[1]])

在TensorFlow中计算Kronecker产品的最有效方法是什么？

4 个答案: