Question

我正在尝试使用python实现梯度下降算法，以下是我的代码，

def grad_des(xvalues, yvalues, R=0.01, epsilon = 0.0001, MaxIterations=1000):
    xvalues= np.array(xvalues)
    yvalues = np.array(yvalues)
    length = len(xvalues)
    alpha = 1
    beta = 1
    converged = False
    i=0
    cost = sum([(alpha + beta*xvalues[i] - yvalues[i])**2 for i in range(length)]) / (2 * length)
    start_time = time.time()
    while not converged:      
        alpha_deriv = sum([(alpha + beta*xvalues[i] - yvalues[i]) for i in range(length)]) / (length)
        beta_deriv =  sum([(alpha + beta*xvalues[i] - yvalues[i])*xvalues[i] for i in range(length)]) / (length)
        alpha = alpha - R * alpha_deriv
        beta = beta - R * beta_deriv
        new_cost = sum( [ (alpha + beta*xvalues[i] - yvalues[i])**2 for i in range(length)] )  / (2*length)
        if abs(cost - new_cost) <= epsilon:
            print 'Converged'
            print 'Number of Iterations:', i
            converged = True
        cost = new_cost
        i = i + 1      
        if i == MaxIterations:
            print 'Maximum Iterations Exceeded'
            converged = True
    print "Time taken: " + str(round(time.time() - start_time,2)) + " seconds"
    return alpha, beta

此代码工作正常。但问题是，大约600次迭代需要超过25秒。我觉得这不够有效，我尝试在进行计算之前将其转换为数组。这确实将时间从300秒减少到25秒。我觉得它可以减少。有人可以帮我改进这个算法吗？

由于

Answer 1

正如我评论的那样，我无法重现这种缓慢，但这里有一些潜在的问题：

看起来length没有变化，但您反复调用range(length)。在Python 2.x中，range创建了一个列表，反复执行此操作可能会减慢速度（对象创建并不便宜。）使用xrange（或导入与Py3兼容的迭代器range来自six或future）并预先创建范围而不是每次。
i在这里被重用，可能会导致问题。您尝试将其用作整体迭代计数，但使用i的每个列表推导将覆盖函数范围内的i，这意味着＆＃34;迭代＆＃34; count总是以length - 1结束。

Answer 2

我能看到的最低悬的果实是矢量化。你有很多列表理解;它们比for循环更快，但没有正确使用numpy数组。

def grad_des_vec(xvalues, yvalues, R=0.01, epsilon=0.0001, MaxIterations=1000):
    xvalues = np.array(xvalues)
    yvalues = np.array(yvalues)
    length = len(xvalues)
    alpha = 1
    beta = 1
    converged = False
    i = 0
    cost = np.sum((alpha + beta * xvalues - yvalues)**2) / (2 * length)
    start_time = time.time()
    while not converged:
        alpha_deriv = np.sum(alpha + beta * xvalues - yvalues) / length
        beta_deriv = np.sum(
            (alpha + beta * xvalues - yvalues) * xvalues) / length
        alpha = alpha - R * alpha_deriv
        beta = beta - R * beta_deriv
        new_cost = np.sum((alpha + beta * xvalues - yvalues)**2) / (2 * length)
        if abs(cost - new_cost) <= epsilon:
            print('Converged')
            print('Number of Iterations:', i)
            converged = True
        cost = new_cost
        i = i + 1
        if i == MaxIterations:
            print('Maximum Iterations Exceeded')
            converged = True
    print("Time taken: " + str(round(time.time() - start_time, 2)) + " seconds")
    return alpha, beta

进行比较

In[47]: grad_des(xval, yval)
Converged
Number of Iterations: 198
Time taken: 0.66 seconds
Out[47]: 
(0.28264882215511067, 0.53289263416071131)

In [48]: grad_des_vec(xval, yval)
Converged
Number of Iterations: 198
Time taken: 0.03 seconds
Out[48]: 
(0.28264882215511078, 0.5328926341607112)

大约是20倍速（xval和yval都是1024个元素阵列。）。

梯度下降算法需要很长时间才能完成 - 效率 - Python

2 个答案: