Question

我有一个数据框，其中包含日期时间作为索引，以及一个额外的分组变量status。 TUFNWGTP是一个权重，用于跨群体进行比较

            status      shopping        TUFNWGTP
TUDIARYDATE                                     
2003-01-03     emp  0.000000e+00  8155462.672158
2003-01-04     emp  0.000000e+00  1735322.527819
2003-01-04     emp  7.124781e+09  3830527.482672
2003-01-02   unemp  0.000000e+00  6622022.995205
2003-01-09     emp  0.000000e+00  3068387.344956

当我试图按月累计一个月时，我正在做

test = dfNew.groupby([pd.TimeGrouper("QS", label='left'), 'status']).sum()
result = pd.DataFrame(test['shopping']/test['TUFNWGTP'], columns=['shopping_weighted'])
result.unstack().plot()

quarterly aggregation

对于实时系列比较，这些波动太大。然后我做了同样的练习，按月分组：

test2 = dfNew.groupby([pd.TimeGrouper("AS", label='left'), 'status']).sum()
result2 = pd.DataFrame(test2['shopping']/test2['TUFNWGTP'], columns=['shopping_weighted'])
result2.unstack().plot()
plt.show()

annual aggregation

仍然尖刻。现在，我想为状态为的每个组计算滚动窗口。我试图首先计算季度窗口，然后创建超过12个月的滚动平均值：

pd.stats.moments.rolling_mean(test['shopping']/test['TUFNWGTP'], 12).unstack().plot() plt.show()

这让我更清楚地向下倾斜。但是，这会给我两个时间序列，对于两个不同的status组看起来非常相似，我认为pandas在某种程度上是对各组进行平均。我该怎么办？

以下是您自己复制的一些数据 - 它是用于第一个图表的季度汇总数据（test）：

shopping TUFNWGTP TUDIARYDATE status 2003-01-01 emp 8.292987e+12 1.265939e+10 unemp 8.920840e+11 1.175799e+09 2003-04-01 emp 9.253035e+12 1.338543e+10 unemp 7.551139e+11 1.131358e+09 2003-07-01 emp 9.237080e+12 1.375033e+10 unemp 7.440140e+11 1.004834e+09 2003-10-01 emp 1.064579e+13 1.339203e+10 unemp 1.061342e+12 1.080896e+09 2004-01-01 emp 8.562482e+12 1.284793e+10 unemp 8.235667e+11 1.169355e+09 2004-04-01 emp 8.773047e+12 1.326451e+10 unemp 5.907015e+11 1.093678e+09 2004-07-01 emp 9.479579e+12 1.350767e+10 unemp 1.115300e+12 1.162550e+09 2004-10-01 emp 1.136157e+13 1.375178e+10 unemp 8.104915e+11 8.251867e+08 2005-01-01 emp 8.105330e+12 1.351932e+10 unemp 6.082188e+11 1.064661e+09 2005-04-01 emp 9.176033e+12 1.358672e+10 unemp 8.631214e+11 9.917538e+08 2005-07-01 emp 9.937520e+12 1.414141e+10 unemp 6.275015e+11 8.850640e+08 2005-10-01 emp 1.044345e+13 1.378072e+10 unemp 9.742346e+11 9.248803e+08 2006-01-01 emp 9.533602e+12 1.349918e+10 unemp 5.105317e+11 9.877952e+08 2006-04-01 emp 8.446490e+12 1.349727e+10 unemp 8.582609e+11 1.007284e+09 2006-07-01 emp 9.167158e+12 1.404490e+10 unemp 8.219319e+11 9.176818e+08 2006-10-01 emp 1.188230e+13 1.413748e+10 unemp 1.641259e+12 1.058742e+09 2007-01-01 emp 9.410542e+12 1.408026e+10 unemp 5.747821e+11 8.084116e+08 2007-04-01 emp 9.492969e+12 1.401190e+10 unemp 4.231717e+11 9.895104e+08 2007-07-01 emp 9.602594e+12 1.417303e+10 unemp 7.458046e+11 9.295575e+08 2007-10-01 emp 1.106523e+13 1.449304e+10 unemp 1.204043e+12 1.112283e+09

Answer 1

你是对的

pd.stats.moments.rolling_mean(test['shopping']/test['TUFNWGTP'], 12).unstack().plot()

混合来自两组的值。您可以看到前11行是NaN，而不管status：

In [82]: pd.stats.moments.rolling_mean(test['shopping']/test['TUFNWGTP'], 12)
Out[82]: 
            status
2003-01-01  emp            NaN
            unemp          NaN
2003-04-01  emp            NaN
            unemp          NaN
2003-07-01  emp            NaN
            unemp          NaN
2003-10-01  emp            NaN
            unemp          NaN
2004-01-01  emp            NaN
            unemp          NaN
2004-04-01  emp            NaN
            unemp     1.078546
2004-07-01  emp       1.077651
            unemp     1.086730
2004-10-01  emp       1.050206

因此，不要先使用test，而是首先取消堆叠test，这样您就会得到两列 - 一列用于emp，另一列用于unemp：

result = pd.DataFrame(
    test['shopping']/test['TUFNWGTP'], columns=['shopping_weighted'])
result = result.unstack()
print(result.head())

产量

           shopping_weighted          
status                   emp     unemp
2003-01-01          1.100091  0.871605
2003-04-01          1.188454  1.369590
2003-07-01          0.987842  1.103778
2003-10-01          0.888269  1.133720
2004-01-01          0.950096  1.239608

然后将rolling_mean应用于result，这样就可以获得两列滚动方式：

In [94]: pd.stats.moments.rolling_mean(result, 12).head(20)
Out[94]: 
           shopping_weighted          
status                   emp     unemp
...
2005-07-01               NaN       NaN
2005-10-01          0.994440  1.109355
2006-01-01          0.978686  1.128826
2006-04-01          0.964123  1.104678
2006-07-01          0.961347  1.104975
2006-10-01          0.971852  1.111623
2007-01-01          0.973510  1.085946
2007-04-01          0.986782  1.080206
2007-07-01          0.990422  1.095752
2007-10-01          1.006258  1.077732

例如，

import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
np.random.seed(1)


dates = pd.date_range('2003-01-03', '2015-03-01', freq='D')
N = len(dates)
index = sorted(np.random.choice(dates, N, replace=True))
status = np.random.choice(['emp', 'unemp'], N, replace=True)
shopping = np.random.random(N)
TUFNWGTP = np.random.random(N)
dfNew = pd.DataFrame({'status': status, 'shopping': shopping, 'TUFNWGTP': TUFNWGTP},
                     index=dates)
mask = dfNew['status'] == 'unemp'
dfNew.loc[mask, 'shopping'] *= 1.1
test = dfNew.groupby([pd.TimeGrouper("QS", label='left'), 'status']).sum()
result = pd.DataFrame(
    test['shopping']/test['TUFNWGTP'], columns=['shopping_weighted'])
result = result.unstack()
pd.stats.moments.rolling_mean(result, 12).plot()
plt.show()

的产率 enter image description here

滚动窗口为不同的组

1 个答案: