Question

我想通过||Ah(Wx)-y||我们ReLU

的梯度下降最小化h

s = 9
n = 99
m = 999
A = np.random.normal((n,m))
y = np.random.normal((m,1))
W = np.random.normal((n,s))

def obj_fcn(x):
    return np.linalg.norm(A.dot(np.max(W.dot(x),0))-y)

if __name__ == '__main__':
    dx = 10
    max_iter = 99
    i = 0
    x = np.zeros((s,1))
    ddx = 0.00001
    ddx2 = ddx*2
    while i<max_iter:
        d_obj = (obj_fcn(x+ddx)-obj_fcn(x-ddx))/ddx2
        x = x + d_obj
        print(x)
        i = i+1

这会产生错误

/Library/Frameworks/Python.framework/Versions/3.6/bin/python3.6 /Users/Chu/Documents/fun/m608.py
Traceback (most recent call last):
  File "/Users/Chu/Documents/fun/m608.py", line 21, in <module>
    d_obj = (obj_fcn(x+ddx)-obj_fcn(x-ddx))/ddx2
  File "/Users/Chu/Documents/fun/m608.py", line 11, in obj_fcn
    return np.linalg.norm(A.dot(np.max(W.dot(x),0))-y)
ValueError: shapes (2,) and (9,1) not aligned: 2 (dim 0) != 9 (dim 0)

怎么了？我能读一本关于Python中这类问题的书吗？我几乎总是遇到这种麻烦。

Answer 1

我将您的功能分解为单独的操作，并在计算后立即打印输入和结果。这是一种非常基本的调试技术;如果您要在琐碎的应用程序之外的任何级别进行编程，那么需要才能学习此技能。查看这个可爱的debug博客以获取帮助。

def obj_fcn(x):
    print("x", x)
    print("W", W)
    wdot = W.dot(x)
    print("wdot", wdot)
    wdot = np.max(wdot, 0)
    adot = A.dot(wdot)
    print("adot", adot)
    norm_arg = adot - y   
    print("arg", norm_arg)
    result = np.linalg.norm(norm_arg)
    return result
    # return np.linalg.norm(A.dot(np.max(W.dot(x),0))-y)

输出：

x [[  1.00000000e-05]
 [  1.00000000e-05]
 [  1.00000000e-05]
 [  1.00000000e-05]
 [  1.00000000e-05]
 [  1.00000000e-05]
 [  1.00000000e-05]
 [  1.00000000e-05]
 [  1.00000000e-05]]
W [ 98.90267125   7.01706833]
Traceback (most recent call last):
  File "so.py", line 32, in <module>
    d_obj = (obj_fcn(x+ddx) - obj_fcn(x-ddx)) / ddx2
  File "so.py", line 13, in obj_fcn
    wdot = W.dot(x)
ValueError: shapes (2,) and (9,1) not aligned: 2 (dim 0) != 9 (dim 0)

这清楚地显示了错误：您正试图获取不同长度项目的点积。没有为这种情况定义点积。您必须修复输入处理。

这种梯度下降有什么问题？

1 个答案: