Question

我在Tensorflow中实现RBF网络时遇到问题。我需要计算x和质心之间的欧几里德距离（来自RBF newtork的定义）。我想这个代码：

    x_data = tf.placeholder(shape=[None, 3], dtype=tf.float32)
    y_target = tf.placeholder(shape=[None, 1], dtype=tf.float32)

    # Create variables for NN layers
    A1 = tf.Variable(tf.random_normal(shape=[3, first_layer_nodes]))  # input -> first layer nodes
    A2 = tf.Variable(tf.random_normal(shape=[first_layer_nodes, 1]))  # first_layer nodes -> sum node
    c = tf.Variable(tf.random_normal(shape=[first_layer_nodes]))  # centroids

    # Declare NN
    inputs_with_weights = tf.matmul(x_data, A1)
    print(inputs_with_weights)
    # euclid_dist = tf.sqrt(tf.reduce_sum(tf.pow(tf.subtract(inputs_with_weights, c), 2)))
    euclid_dist = tf.norm(inputs_with_weights - c, ord='euclidean')
    print(euclid_dist)
    first_output = tf_gaussian_function(euclid_dist)
    print(first_output)

final_output = tf.matmul（first_output，A2）

但我是这样的问题：

E:\#PROJEKTY\#PROGRAMOWANIE\AI-Project>python Iris.py
2018-04-27 00:49:37.800684: I C:\tf_jenkins\workspace\rel-win\M\windows\PY\36\tensorflow\core\platform\cpu_feature_guard.cc:140] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2
<tf.Variable 'Variable_2:0' shape=(1, 1) dtype=float32_ref>
Tensor("MatMul:0", shape=(?, 1), dtype=float32)
Tensor("norm/Squeeze:0", shape=(), dtype=float32)
Tensor("gaussian_function:0", dtype=float32)
Traceback (most recent call last):
  File "C:\Users\Szatku\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\client\session.py", line 1361, in _do_call
    return fn(*args)
  File "C:\Users\Szatku\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\client\session.py", line 1340, in _run_fn
    target_list, status, run_metadata)
  File "C:\Users\Szatku\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\framework\errors_impl.py", line 516, in __exit__
    c_api.TF_GetCode(self.status.status))
tensorflow.python.framework.errors_impl.InvalidArgumentError: In[0] is not a matrix
         [[Node: MatMul_1 = MatMul[T=DT_FLOAT, transpose_a=false, transpose_b=false, _device="/job:localhost/replica:0/task:0/device:CPU:0"](gaussian_function, Variable_1/read)]]

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "Iris.py", line 144, in <module>
    sess.run(train_step, feed_dict={x_data: x_d, y_target: y_d})
  File "C:\Users\Szatku\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\client\session.py", line 905, in run
    run_metadata_ptr)
  File "C:\Users\Szatku\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\client\session.py", line 1137, in _run
    feed_dict_tensor, options, run_metadata)
  File "C:\Users\Szatku\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\client\session.py", line 1355, in _do_run
    options, run_metadata)
  File "C:\Users\Szatku\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\client\session.py", line 1374, in _do_call
    raise type(e)(node_def, op, message)
tensorflow.python.framework.errors_impl.InvalidArgumentError: In[0] is not a matrix
         [[Node: MatMul_1 = MatMul[T=DT_FLOAT, transpose_a=false, transpose_b=false, _device="/job:localhost/replica:0/task:0/device:CPU:0"](gaussian_function, Variable_1/read)]]

Caused by op 'MatMul_1', defined at:
  File "Iris.py", line 124, in <module>
    final_output = tf.matmul(first_output, A2)
  File "C:\Users\Szatku\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\ops\math_ops.py", line 2064, in matmul
    a, b, transpose_a=transpose_a, transpose_b=transpose_b, name=name)
  File "C:\Users\Szatku\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\ops\gen_math_ops.py", line 2790, in _mat_mul
    name=name)
  File "C:\Users\Szatku\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\framework\op_def_library.py", line 787, in _apply_op_helper
    op_def=op_def)
  File "C:\Users\Szatku\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\framework\ops.py", line 3271, in create_op
    op_def=op_def)
  File "C:\Users\Szatku\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\framework\ops.py", line 1650, in __init__
    self._traceback = self._graph._extract_stack()  # pylint: disable=protected-access

InvalidArgumentError (see above for traceback): In[0] is not a matrix
         [[Node: MatMul_1 = MatMul[T=DT_FLOAT, transpose_a=false, transpose_b=false, _device="/job:localhost/replica:0/task:0/device:CPU:0"](gaussian_function, Variable_1/read)]]

如果我尝试将任何值放在轴上，它会停留在：

E:\#PROJEKTY\#PROGRAMOWANIE\AI-Project>python Iris.py
2018-04-27 00:53:15.388129: I C:\tf_jenkins\workspace\rel-win\M\windows\PY\36\tensorflow\core\platform\cpu_feature_guard.cc:140] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2
<tf.Variable 'Variable_2:0' shape=(1, 1) dtype=float32_ref>
Tensor("MatMul:0", shape=(?, 1), dtype=float32)
Traceback (most recent call last):
  File "C:\Users\Szatku\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\framework\common_shapes.py", line 686, in _call_cpp_shape_fn_impl
    input_tensors_as_shapes, status)
  File "C:\Users\Szatku\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\framework\errors_impl.py", line 516, in __exit__
    c_api.TF_GetCode(self.status.status))
tensorflow.python.framework.errors_impl.InvalidArgumentError: Invalid reduction dimension 2 for input with 2 dimensions. for 'norm/Sum' (op: 'Sum') with input shapes: [?,1], [1] and with computed input tensors: input[1] = <2>.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "Iris.py", line 120, in <module>
    euclid_dist = tf.norm(inputs_with_weights - c, axis = 2, ord='euclidean')
  File "C:\Users\Szatku\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\util\deprecation.py", line 432, in new_func
    return func(*args, **kwargs)
  File "C:\Users\Szatku\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\ops\linalg_ops.py", line 552, in norm
    tensor * math_ops.conj(tensor), axis, keepdims=True))
  File "C:\Users\Szatku\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\util\deprecation.py", line 432, in new_func
    return func(*args, **kwargs)
  File "C:\Users\Szatku\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\ops\math_ops.py", line 1373, in reduce_sum
    name=name))
  File "C:\Users\Szatku\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\ops\gen_math_ops.py", line 5436, in _sum
    name=name)
  File "C:\Users\Szatku\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\framework\op_def_library.py", line 787, in _apply_op_helper
    op_def=op_def)
  File "C:\Users\Szatku\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\framework\ops.py", line 3273, in create_op
    compute_device=compute_device)
  File "C:\Users\Szatku\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\framework\ops.py", line 3313, in _create_op_helper
    set_shapes_for_outputs(op)
  File "C:\Users\Szatku\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\framework\ops.py", line 2501, in set_shapes_for_outputs
    return _set_shapes_for_outputs(op)
  File "C:\Users\Szatku\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\framework\ops.py", line 2474, in _set_shapes_for_outputs
    shapes = shape_func(op)
  File "C:\Users\Szatku\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\framework\ops.py", line 2404, in call_with_requiring
    return call_cpp_shape_fn(op, require_shape_fn=True)
  File "C:\Users\Szatku\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\framework\common_shapes.py", line 627, in call_cpp_shape_fn
    require_shape_fn)
  File "C:\Users\Szatku\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\framework\common_shapes.py", line 691, in _call_cpp_shape_fn_impl
    raise ValueError(err.message)
ValueError: Invalid reduction dimension 2 for input with 2 dimensions. for 'norm/Sum' (op: 'Sum') with input shapes: [?,1], [1] and with computed input tensors: input[1] = <2>.

我不知道如何更正代码。有人可以帮助我吗？

编辑：高斯实现：

def gaussian_function(input_layer):
    initial = math.exp(-SC*math.pow(input_layer, 2))
    return initial


np_gaussian_function = np.vectorize(gaussian_function)


def d_gaussian_function(input_layer):
    initial = -2*SC*input_layer * math.exp(-SC * math.pow(input_layer, 2))
    return initial


np_d_gaussian_function = np.vectorize(d_gaussian_function)


def np_d_gaussian_function_32(input_layer):
    return np_d_gaussian_function(input_layer).astype(np.float32)


def tf_d_gaussian_function(input_layer, name=None):
    with ops.name_scope(name, "d_gaussian_function", [input_layer]) as name:
        y = tf.py_func(np_d_gaussian_function_32, [input_layer],[tf.float32], name=name, stateful=False)
    return y[0]


def py_func(func, inp, Tout, stateful=True, name=None, grad=None):
    rnd_name = 'PyFunGrad' + str(np.random.randint(0, 1E+8))

    tf.RegisterGradient(rnd_name)(grad)
    g = tf.get_default_graph()
    with g.gradient_override_map({"PyFunc": rnd_name}):
        return tf.py_func(func, inp, Tout, stateful=stateful, name=name)


def gaussian_function_grad(op, grad):
    input_variable = op.inputs[0]
    n_gr = tf_d_gaussian_function(input_variable)
    return grad * n_gr


def np_gaussian_function_32(input_layer):
    return np_gaussian_function(input_layer).astype(np.float32)


def tf_gaussian_function(input_layer, name=None):
    with ops.name_scope(name, "gaussian_function", [input_layer]) as name:
        y = py_func(np_gaussian_function_32, [input_layer], [tf.float32], name=name, grad=gaussian_function_grad)
    return y[0]
# end of defining activation function

Answer 1

错误表示

的第一个参数

final_output = tf.matmul(first_output, A2)

不是矩阵。所以first_output不合适。它来自上一行：

first_output = tf_gaussian_function(euclid_dist)

因此我们需要检查该函数tf_gaussian_function()，但这不在发布的代码中。我已经检查过它是否是一个拼写错误，它应该是tf.gaussian_function（）但是没有这样的东西。

所以请发布函数tf_gaussian_function()的定义。

现在我们已经知道了，事实证明你正试图将函数math.exp( -SC * math.pow( input_layer, 2 ) )（通常称为rho或 ρ ）映射到{ {1}};我认为这将是你的径向基函数。旁注：看起来你正试图通过实现自己的渐变来完成一些严重的繁重工作，以及所有这些，通过euclid_dist以及所有这些。我建议你实现一些更简单的东西，比如这段代码（未经测试）：

np.vectorize()

（我使用first_output = tf.exp( -SC * euclid_dist * euclid_dist )代替等效的x * x，因为第一个效率更高。）这应该创建自己的渐变，并照顾你想要实现的一切。但无论如何，你确信你的实现是有效的，这可能就是我发现的here;你也可以仔细检查它是否产生与这个单线相同的结果。

就tf.pow( x, 2 )而言，tf.norm( x, ord = "euclidean", axis = None )返回2 ^nd范数（或欧几里德范数，否则也称为平方和的平方根），所以这绝对是一个标量。（它完全上面注释掉的行：euclid_dist。）不确定你想要在那里实现什么？如果你只想要距离的平方，那么你应该这样做：

euclid_dist = tf.sqrt(tf.reduce_sum(tf.pow(tf.subtract(inputs_with_weights, c), 2)))

但据我所知，在径向基函数网络中，首先在输入上应用权重，然后是径向基函数，并且仅作为第三步，将欧几里德距离作为成本函数，这可能非常好吧是一个标量。

如果您试图为RBF网络实施 规范化 ，请注意normalization is generally done有点不同。因此，如果我抓住你的漂移，你的实现可能看起来更像这样：

euclid_dist = ( inputs_with_weights - c ) * ( inputs_with_weights - c )

输出：

Tensor（“MatMul_15：0”，shape =（1,5），dtype = float32）
  张量（“sub_8：0”，shape =（1,5），dtype = float32）
  张量（“div_5：0”，形状=（1,5），dtype = float32）
  [[4.4366708e-03 6.8647589e-04 5.9621310e-01 7.5066246e-06 3.9865622e-01]]
  [[0.31285414]]

要回复您发布此代码片段的评论：

from __future__ import print_function
import tensorflow as tf
from tensorflow.python.framework import ops
import numpy as np

x_data = tf.placeholder(shape=[1, 3], dtype=tf.float32)
y_target = tf.placeholder(shape=[1, 1], dtype=tf.float32)

# Create variables for NN layers
first_layer_nodes = 5
A1 = tf.Variable(tf.random_normal(shape=[3, first_layer_nodes]))  # input -> first layer nodes
A2 = tf.Variable(tf.random_normal(shape=[first_layer_nodes, 1]))  # first_layer nodes -> sum node
c = tf.Variable(tf.random_normal(shape=[first_layer_nodes]))  # centroids

# Declare NN
SC = 1
def rho( distances ): return tf.exp( -SC * distances * distances )
def norm( x ): return x / tf.reduce_sum( x, axis = -1 )
inputs_with_weights = tf.matmul( x_data, A1 )
print(inputs_with_weights)
distances = inputs_with_weights - c
print( distances )
first_output = norm( rho( distances ) ) # tf_gaussian_function(distances) # 
print(first_output)
final_output = tf.matmul(first_output, A2)

with tf.Session() as sess:
    sess.run( tf.global_variables_initializer() )
    r = sess.run( [ first_output, final_output ], feed_dict = {
                  x_data : np.array( [ [ 1.0, 2, 3 ] ] ) } )
    for v in r:
        print( v )

这可以进行矢量化，利用tf.subtract()上的隐式广播和类似的高级索引（未经测试）：

exp_list = []
for i in range(first_layer_nodes):
    euclid_distance = tf.reduce_sum(tf.square(tf.subtract(x_data, c[i, :])), 1)
    exp_list.append(tf.exp(-SC * euclid_distance))
phi = tf.transpose(tf.stack(exp_list))

Tensorflow - 计算RBF网络的欧氏距离

1 个答案: