Sklearn线性回归损失函数与手册代码不匹配

Question

我一直在尝试按照Sklearn线性回归库的手册代码复制cost的结果。两者之间存在巨大差异，我无法弄清原因。这是来自Sklearn的代码：

SkLearn实施：

X_train, X_test, Y_train, Y_test = model_selection.train_test_split(X, Y, test_size=0.30)
classifier = sklearn.linear_model.LinearRegression()
classifier.fit(X_train,Y_train)
cost = np.sqrt(np.sum((np.dot(X_train,classifier.coef_.reshape(9,1)) + classifier.intercept_ - Y_train.reshape(478,1))**2))
print(cost)

cost = 4.236441942240197

我尝试复制结果：

a = X_train_rev.shape
assert(X_train_rev.shape == (478,10)) # assert shape of the X_train_rev
Y_train = Y_train.reshape(478,1)

alpha = 0.0005 # Learning_Rate
coefficient = np.random.randn(1,10) # Initialisation of coefficients including intercept

# Loop through iterations
for i in range(100000):
    cost = np.sqrt(np.sum((np.dot(X_train_rev,coefficient.T) - Y_train)**2)) # cost result
    if i % 10000 == 0: print(cost)
    grad = np.dot((np.dot(X_train_rev,coefficient.T) - Y_train).T, X_train_rev) # Compute Gradients
    coefficient = coefficient - (alpha * grad) # adjust coefficients including intercept



Cost after Iterations:
45.23042864973579
10.428401916963285
10.428401916963285
10.428401916963285
10.428401916963285
10.428401916963285
10.428401916963285
10.428401916963285
10.428401916963285
10.428401916963285

根据我的手册代码，成本并没有进一步降低，并且与Sklearn的成本相差甚远。我尝试使用alpha变量，但是alpha的任何增加都会导致成本趋于正无穷大。

请注意，在我的手动代码中使用的X_train_rev数据具有10列/功能，而不是Sklearn训练集中的9个功能，因为我在训练集中添加了“一”列来表示截距。同样，系数向量也包含拦截。

Answer 1

我试图重复您的问题

from sklearn.datasets import make_regression
from sklearn.model_selection import train_test_split
from sklearn.linear_model import LinearRegression
from sklearn.metrics import mean_squared_error

X, Y = make_regression(n_samples=500, n_features=9, bias=0, random_state=1)

X_train, X_test, Y_train, Y_test = train_test_split(X, Y, test_size=0.30, random_state=1)
classifier = LinearRegression(fit_intercept=False)
classifier.fit(X_train,Y_train)
cost = np.sqrt(np.sum((np.dot(X_train,classifier.coef_.reshape(9,1)) + classifier.intercept_ - Y_train.reshape(-1,1))**2))

print(cost)
print('Manual regression')
Y_train = Y_train.reshape(-1,1)

alpha = 0.0005 # Learning_Rate
coefficient = np.random.randn(1,9) # Initialisation of coefficients including intercept

# Loop through iterations
for i in range(100000):
    cost = np.sqrt(np.sum((np.dot(X_train,coefficient.T) - Y_train)**2)) # cost result
    if i % 10000 == 0: print(cost)
    grad = np.dot((np.dot(X_train,coefficient.T) - Y_train).T, X_train) # Compute Gradients
    coefficient = coefficient - (alpha * grad) # adjust coefficients including intercept

进行细微调整以使代码完全可复制。而且我没有遇到任何问题。 MSE的差异很小，但两个分数均小于1e-11，因此这是一个数字问题。