Python中的约束回归

Question

我有这个简单的回归模型：

 y = a + b * x + c * z + error

对参数有约束：

c = b - 1

在SO上发布了类似的问题（如Constrained Linear Regression in Python）。但是，约束＆＃39;类型为lb <= parameter =< ub。

处理此特定约束线性回归问题的可用选项有哪些？

Answer 1

这是使用GLM完成的方法：

import statsmodels
import statsmodels.api as sm
import numpy as np

# Set the link function to identity
statsmodels.genmod.families.links.identity()

OLS_from_GLM = sm.GLM(y, sm.add_constant(np.column_stack(x, z)))

 '''Setting the restrictions on parameters in the form of (R, q), where R 
 and q are constraints' matrix and constraints' values, respectively. As
 for the restriction in the aforementioned regression model, i.e., 
 c = b - 1 or b - c = 1, R = [0, 1, -1] and q = 1.'''

res_OLS_from_GLM = OLS_from_GLM.fit_constrained(([0, 1.0, -1.0], 1))

print(res_OLS_from_GLM.summary())

Answer 2

Python 中有一些约束优化包，如 CVX、CASADI、GEKKO、Pyomo 等，可以解决这个问题。我为具有微分或代数约束的线性、非线性和混合整数优化问题开发了 Gekko。

import numpy as np
from gekko import GEKKO

# Data
x = np.random.rand(10)
y = np.random.rand(10)
z = np.random.rand(10)

# Gekko for constrained regression
m = GEKKO(remote=False); m.options.IMODE=2
a,b,c  = m.Array(m.FV,3)
a.STATUS=1; b.STATUS=1; c.STATUS=1
x=m.Param(x); z=m.Param(z)
y = m.Var();  ym=m.Param(y)
m.Equation(y==a+b*x+c*z)
m.Equation(c==b-1)
m.Minimize((ym-y)**2)
m.options.SOLVER=1
m.solve(disp=True)
print(a.value[0],b.value[0],c.value[0])

这给出的解决方案在您运行时可能会有所不同，因为它对数据使用随机值。

-0.021514129645 0.45830726553 -0.54169273447

约束 c = b - 1 满足 -0.54169273447 = 0.45830726553 - 1。以下是与其他无约束的 linear regression packages in Python 的比较：

import numpy as np
from scipy.stats import linregress
import statsmodels.api as sm
import matplotlib.pyplot as plt
from gekko import GEKKO

# Data
x = np.array([4,5,2,3,-1,1,6,7])
y = np.array([0.3,0.8,-0.05,0.1,-0.8,-0.5,0.5,0.65])

# calculate R^2
def rsq(y1,y2):
    yresid= y1 - y2
    SSresid = np.sum(yresid**2)
    SStotal = len(y1) * np.var(y1)
    r2 = 1 - SSresid/SStotal
    return r2

# Method 1: scipy linregress
slope,intercept,r,p_value,std_err = linregress(x,y)
a = [slope,intercept]
print('R^2 linregress = '+str(r**2))

# Method 2: numpy polyfit (1=linear)
a = np.polyfit(x,y,1); print(a)
yfit = np.polyval(a,x)
print('R^2 polyfit    = '+str(rsq(y,yfit)))

# Method 3: numpy linalg solution
#       y =     X a
#   X^T y = X^T X a
X = np.vstack((x,np.ones(len(x)))).T
# matrix operations
XX = np.dot(X.T,X)
XTy = np.dot(X.T,y)
a = np.linalg.solve(XX,XTy)
# same solution with lstsq
a = np.linalg.lstsq(X,y,rcond=None)[0]
yfit = a[0]*x+a[1]; print(a)
print('R^2 matrix     = '+str(rsq(y,yfit)))

# Method 4: statsmodels ordinary least squares
X = sm.add_constant(x,prepend=False)
model = sm.OLS(y,X).fit()
yfit = model.predict(X)
a = model.params
print(model.summary())

# Method 5: Gekko for constrained regression
m = GEKKO(remote=False); m.options.IMODE=2
c  = m.Array(m.FV,2); c[0].STATUS=1; c[1].STATUS=1
c[1].lower=-0.5
xd = m.Param(x); yd = m.Param(y); yp = m.Var()
m.Equation(yp==c[0]*xd+c[1])
m.Minimize((yd-yp)**2)
m.solve(disp=False)
c = [c[0].value[0],c[1].value[1]]
print(c)

# plot data and regressed line
plt.plot(x,y,'ko',label='data')
xp = np.linspace(-2,8,100)
slope     = str(np.round(a[0],2))
intercept = str(np.round(a[1],2))
eqn = 'LstSQ: y='+slope+'x'+intercept
plt.plot(xp,a[0]*xp+a[1],'r-',label=eqn)
slope     = str(np.round(c[0],2))
intercept = str(np.round(c[1],2))
eqn = 'Constraint: y='+slope+'x'+intercept
plt.plot(xp,c[0]*xp+c[1],'b--',label=eqn)
plt.grid()
plt.legend()
plt.show()