使用标准定标器进行梯度下降收敛的原因

Question

我有两个查询

sklearn标准缩放器StandardScaler（）前后的系数解释是否相同

使用标准缩放器后，梯度下降和sklearn合成系数相同，情况将如何变化。

标准缩放器会将数据转换为零和单位方差的平均值

下面的代码显示了示例

在应用标准缩放器之前，梯度下降和sklearn给出不同的系数，因为梯度下降陷入局部最小值，但是在应用标准缩放器之后，两者均收敛到相同的系数

代码实现：

import numpy as np
from sklearn.linear_model import LinearRegression
from sklearn.preprocessing import StandardScaler

IV1 = np.array([63 , 64, 66, 69, 69, 71, 71, 72, 73, 75])
DV = np.array([127,121,142,157,162,156,169,165,181,208])

X = IV1[:,np.newaxis]
Y = DV[:,np.newaxis]

def compute_gradient(m ,b ,IV, DV, cost):

    gradient_b = 0
    gradient_m = 0
    for i in range (data_size):
    x = IV[i]
    y = DV[i]
    gradient_b += (-2/data_size) * (y - (m * x + b))
    gradient_m += (-2/data_size) * x * (y- (m * x + b))

    new_m = m - (gradient_m * learning_rate)
    new_b = b - (gradient_b * learning_rate)

    gradient_m = -gradient_m
    cost.append (gradient_m)

    return (new_m,new_b)




data_size = X.size

iterations = 10000
initial_b  = 0 
initial_m  = 0
learning_rate = 0.0001


m = initial_m
b = initial_b
cost = []

for i in range (iterations):
    [m,b] = compute_gradient(m ,b , IV1, DV, cost) 

print ("*******************************") 
print (" Gradient Descent")   
print ("iteration {} m {} b {}".format(i, m, b))
print ("******************************* \n")


lm = LinearRegression()
lm.fit(X,Y)

print ("*******************************")
print ("From Sklearn")
print ("Coeff {}".format(lm.coef_)) 
print ("Intercept {}".format(lm.intercept_)) 
print ("******************************* \n")


iterations = 10000
initial_b  = 0 
initial_m  = 0
learning_rate = 0.001

sc_x = StandardScaler()
sc_y = StandardScaler()
X_std = sc_x.fit_transform(X)
#y_std = sc_y.fit_transform(Y)
y_std = Y

m = initial_m
b = initial_b
cost = []

for i in range (iterations):
    [m,b] = compute_gradient(m ,b , X_std, y_std, cost) 

print ("*******************************") 
print (" Gradient Descent after standardization")       
print ("iteration {} m {} b {}".format(i, m, b))
print ("******************************* \n")


lm = LinearRegression()
lm.fit(X_std,y_std)

print ("*******************************")
print ("From Sklearn")
print ("Coeff {}".format(lm.coef_)) 
print ("Intercept {}".format(lm.intercept_))
print ("*******************************") 

结果：

*******************************
 Gradient Descent
iteration 9999 m 2.323963705514206 b -1.490870527668376
******************************* 

*******************************
From Sklearn
Coeff [[6.13758146]]
Intercept [-266.53439537]
******************************* 

*******************************
Gradient Descent after standardization
iteration 9999 m [22.80836256] b [158.79999968]
******************************* 

*******************************
From Sklearn
Coeff [[22.80836261]]
Intercept [158.8]
*******************************