子集数据点在置信区间之外

Question

使用与之前的question相同的示例（下面粘贴的代码），我们可以使用statsmodels outliers_influence中的summary_table函数获取95％CI。但现在，如何仅对置信区间之外的数据点（x和y）进行子集化？

import numpy as np
import statsmodels.api as sm
from statsmodels.stats.outliers_influence import summary_table

#measurements genre
n = 100
x = np.linspace(0, 10, n)
e = np.random.normal(size=n)
y = 1 + 0.5*x + 2*e
X = sm.add_constant(x)
re = sm.OLS(y, X).fit()
st, data, ss2 = summary_table(re, alpha=0.05)
predict_ci_low, predict_ci_upp = data[:, 6:8].T

Answer 1

这可能为时已晚，但是您可以将其放入pandas.DataFrame并根据布尔值列表进行过滤。假设我收到您的问题：

import numpy as np
import statsmodels.api as sm
from statsmodels.stats.outliers_influence import summary_table
import matplotlib.pyplot as plot

## Import pandas
import pandas as pd

#measurements genre
n = 100
x = np.linspace(0, 10, n)
e = np.random.normal(size=n)
y = 1 + 0.5*x + 2*e
X = sm.add_constant(x)
re = sm.OLS(y, X).fit()
st, data, ss2 = summary_table(re, alpha=0.05)

# Make prediction
prediction = re.predict(X)
predict_ci_low, predict_ci_upp = data[:, 6:8].T

# Put y and x in a pd.DataFrame
df = pd.DataFrame(y).set_index(x)

# Get the y values that are out of the ci intervals. This could be done directly in the df indexer
out_up = y > predict_ci_upp
out_down = y < predict_ci_low

# Plot everything
plot.plot(x, y, label = 'train')
plot.plot(df[out_up], marker = 'o', linewidth = 0)
plot.plot(df[out_down], marker = 'o', linewidth = 0)
plot.plot(x, predictionTrain, label = 'prediction')
plot.plot(x, predict_ci_upp, label = 'ci_up')
plot.plot(x, predict_ci_low, label = 'ci_low')
plot.legend(loc='best')

这是结果图：