Question

编辑：数据集是Andrew Ng机器学习课程第4周作业的MNIST数据集

我已经检查了question on scipy optimize，但我仍然无法弄清楚我的代码有什么问题。我正在努力为Andrew Ng coursera课程中的oneVsAll问题优化theta。

以下是相关代码

def sigmoid(x):
a = []
for item in x:
    a.append(1/(1+math.exp(-item)))
return a

def hypothesis(x, theta):
    return np.array(sigmoid(np.dot(x, theta)))

def costFunction(theta, x, y, lamba_):
    m = X.shape[0]

    part1 = np.dot(y.T, np.log(hypothesis(x, theta)).reshape(m,1))
    part2 = np.dot((np.ones((m,1)) - y).T, np.log( 1 - hypothesis(x, theta)).reshape(m,1))

    summ = (part1 + part2)

    return -summ[0]/m

def gradientVect(theta, x, y, lambda_):
    n = X.shape[1]
    m = X.shape[0]
    gradient = []

    theta = theta.reshape(n,1)

    beta = hypothesis(x, theta) - y

    reg = theta[1:] * lambda_/m

    grad = np.dot(X.T, beta) * 1./m

    grad[1:] = grad[1:] * reg

    return grad.flatten()


from scipy import optimize

def optimizeTheta(x, y, nLabels, lambda_):

    for i in np.arange(0, nLabels):
        theta = np.zeros((n,1))
        res = optimize.minimize(costFunction, theta, args=(x, (y == i)*1, lambda_), method=None,
                       jac=gradientVect, options={'maxiter':50})
        print(res)
    return result

但正在运行

optimizeTheta(X, y, 10, 0) # X shape = 401, 500

给我以下错误：

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-247-e0e6e4c1eddd> in <module>()
      3 n = X.shape[1]
      4 
----> 5 optimizeTheta(X, y, 10, 0)

<ipython-input-246-0a15e9f4769a> in optimizeTheta(x, y, nLabels, lambda_)
     54         theta = np.zeros((n,1))
     55         res = optimize.minimize(costFunction, x0 = theta, args=(x, (y == i)*1, lambda_), method=None,
---> 56                        jac=gradientVect, options={'maxiter':50})
     57         print(res)
     58     return result

//anaconda/lib/python3.5/site-packages/scipy/optimize/_minimize.py in minimize(fun, x0, args, method, jac, hess, hessp, bounds, constraints, tol, callback, options)
    439         return _minimize_cg(fun, x0, args, jac, callback, **options)
    440     elif meth == 'bfgs':
--> 441         return _minimize_bfgs(fun, x0, args, jac, callback, **options)
    442     elif meth == 'newton-cg':
    443         return _minimize_newtoncg(fun, x0, args, jac, hess, hessp, callback,

//anaconda/lib/python3.5/site-packages/scipy/optimize/optimize.py in _minimize_bfgs(fun, x0, args, jac, callback, gtol, norm, eps, maxiter, disp, return_all, **unknown_options)
    859     gnorm = vecnorm(gfk, ord=norm)
    860     while (gnorm > gtol) and (k < maxiter):
--> 861         pk = -numpy.dot(Hk, gfk)
    862         try:
    863             alpha_k, fc, gc, old_fval, old_old_fval, gfkp1 = \

ValueError: shapes (401,401) and (2005000,) not aligned: 401 (dim 1) != 2005000 (dim 0)

我无法弄清楚为什么形状没有对齐。

谢谢！

Answer 1

所以我意识到我的问题出了什么问题。问题是sigmoid函数返回一个列表而不是一个整数，因此它后来搞乱了矩阵乘法。新的sigmoid函数是

def sigmoid(z):
return(1 / (1 + np.exp(-z)))

Scipy优化函数：矩阵未对齐

1 个答案: