Question

您正在使用statsmodel运行以下模型，它运行正常。

from statsmodels.formula.api import ols
from statsmodels.iolib.summary2 import summary_col #for summary stats of large tables
time_FE_str = ' + C(hour_of_day) + C(day_of_week) + C(week_of_year)'
weather_2_str = ' +  C(weather_index) + rain + extreme_temperature + wind_speed'
model = ols("activity_count ~ C(city_id)"+weather_2_str+time_FE_str, data=df)
results = model.fit()
print summary_col(results).tables

print 'F-TEST:'
hypotheses = '(C(weather_index) = 0), (rain=0), (extreme_temperature=0), (wind_speed=0)'
f_test = results.f_test(hypotheses)

但是，如果我想要包含分类变量C(weather_index)，我不知道如何为F检验制定稀释。我尝试了所有可以想象的版本，但我总是收到错误。

之前有人面对过这个问题吗？

有什么想法吗？

F-TEST:
Traceback (most recent call last):
  File "C:/VK/scripts_python/predict_activity.py", line 95, in <module>
    f_test = results.f_test(hypotheses)
  File "C:\Users\Niko\Anaconda2\envs\gl-env\lib\site-packages\statsmodels\base\model.py", line 1375, in f_test
    invcov=invcov, use_f=True)
  File "C:\Users\Niko\Anaconda2\envs\gl-env\lib\site-packages\statsmodels\base\model.py", line 1437, in wald_test
    LC = DesignInfo(names).linear_constraint(r_matrix)
  File "C:\Users\Niko\Anaconda2\envs\gl-env\lib\site-packages\patsy\design_info.py", line 536, in linear_constraint
    return linear_constraint(constraint_likes, self.column_names)
  File "C:\Users\Niko\Anaconda2\envs\gl-env\lib\site-packages\patsy\constraint.py", line 391, in linear_constraint
    tree = parse_constraint(code, variable_names)
  File "C:\Users\Niko\Anaconda2\envs\gl-env\lib\site-packages\patsy\constraint.py", line 225, in parse_constraint
    return infix_parse(_tokenize_constraint(string, variable_names),
  File "C:\Users\Niko\Anaconda2\envs\gl-env\lib\site-packages\patsy\constraint.py", line 184, in _tokenize_constraint
    Origin(string, offset, offset + 1))
patsy.PatsyError: unrecognized token in constraint
   (C(weather_index) = 0), (rain=0), (extreme_temperature=0), (wind_speed=0)
    ^

Answer 1

方法t_test，wald_test和f_test用于直接对参数进行假设检验，而不是用于整个分类或复合效应。

Results.summary（）显示patsy为分类变量创建的参数名称。这些可用于为分类效果创建对比度或限制。

作为替代，anova_lm直接计算一个术语的假设检验，例如。分类变量无效。

Answer 2

忽略C()！

我尝试对这些数据进行分析。

Area    Clover_yield    Yarrow_stems
A   19.0    220
A   76.7    20
A   11.4    510
A   25.1    40
A   32.2    120
A   19.5    300
A   89.9    60
A   38.8    10
A   45.3    70
A   39.7    290
B   16.5    460
B   1.8 320
B   82.4    0
B   54.2    80
B   27.4    0
B   25.8    450
B   69.3    30
B   28.7    250
B   52.6    20
B   34.5    100
C   49.7    0
C   23.3    220
C   38.9    160
C   79.4    0
C   53.2    120
C   30.1    150
C   4.0 450
C   20.7    240
C   29.8    250
C   68.5    0

当我在代码中显示的ols的第一次调用中使用线性模型时，我最终遇到了您遇到的障碍。但是，当我省略 Area 假设离散水平的事实时，我能够计算出对比的F检验。

import pandas as pd
from statsmodels.formula.api import ols

df = pd.read_csv('clover.csv', sep='\s+')
model = ols('Clover_yield ~ C(Area) + Yarrow_stems', data=df)
model = ols('Clover_yield ~ Area + Yarrow_stems', data=df)

results = model.fit()
print (results.summary())

print (results.f_test(['Area[T.B] = Area[T.C], Yarrow_stems=150']))

这是输出。

请注意，摘要表示可用于制定因子对比的名称，在我们的案例Area[T.B]和Area[T.C]中。

                            OLS Regression Results                            
==============================================================================
Dep. Variable:           Clover_yield   R-squared:                       0.529
Model:                            OLS   Adj. R-squared:                  0.474
Method:                 Least Squares   F-statistic:                     9.726
Date:                Thu, 04 Jan 2018   Prob (F-statistic):           0.000177
Time:                        17:26:03   Log-Likelihood:                -125.61
No. Observations:                  30   AIC:                             259.2
Df Residuals:                      26   BIC:                             264.8
Df Model:                           3                                         
Covariance Type:            nonrobust                                         
================================================================================
                   coef    std err          t      P>|t|      [0.025      0.975]
--------------------------------------------------------------------------------
Intercept       57.5772      6.337      9.086      0.000      44.551      70.603
Area[T.B]        0.3205      7.653      0.042      0.967     -15.411      16.052
Area[T.C]       -0.5432      7.653     -0.071      0.944     -16.274      15.187
Yarrow_stems    -0.1086      0.020     -5.401      0.000      -0.150      -0.067
==============================================================================
Omnibus:                        0.459   Durbin-Watson:                   2.312
Prob(Omnibus):                  0.795   Jarque-Bera (JB):                0.449
Skew:                           0.260   Prob(JB):                        0.799
Kurtosis:                       2.702   Cond. No.                         766.
==============================================================================

Warnings:
[1] Standard Errors assume that the covariance matrix of the errors is correctly specified.
<F test: F=array([[ 27873807.59795523]]), p=4.939796675253845e-83, df_denom=26, df_num=2>

正如我的评论所述，我不清楚你打算测试什么。

修改：在用户333700评论的提示下，我尝试再次运行此代码，其中包含两个截然不同的语句，model = ols('Clover_yield ~ C(Area) + Yarrow_stems', data=df)和print (results.f_test(['C(Area)[T.B] = C(Area)[T.C], Yarrow_stems=150']))。＆＃39; C（地区）[T.B]＆＃39;和＆＃39; C（区域）[T.C]＆＃39;来自改变模型的摘要。因此，对于这种类型的分析，无论是否使用C（）声明都无关紧要。您必须记住使用适当的表单作为虚拟变量，如摘要中所述。

statsmodels patsy假设测试约束'C（）'中的分类变量

2 个答案: