我是Pandas的新手......我收到了大量的民意调查数据;我想计算一个滚动平均值，以便根据三天的窗口得出每天的估计值。据我所知，this question，rolling_ *函数根据指定数量的值计算窗口，而不是特定的日期时间范围。
是否有实现此功能的不同功能？还是我一直在写自己的？
编辑：
示例输入数据：
polls_subset.tail(20)
Out[185]:
favorable unfavorable other
enddate
2012-10-25 0.48 0.49 0.03
2012-10-25 0.51 0.48 0.02
2012-10-27 0.51 0.47 0.02
2012-10-26 0.56 0.40 0.04
2012-10-28 0.48 0.49 0.04
2012-10-28 0.46 0.46 0.09
2012-10-28 0.48 0.49 0.03
2012-10-28 0.49 0.48 0.03
2012-10-30 0.53 0.45 0.02
2012-11-01 0.49 0.49 0.03
2012-11-01 0.47 0.47 0.05
2012-11-01 0.51 0.45 0.04
2012-11-03 0.49 0.45 0.06
2012-11-04 0.53 0.39 0.00
2012-11-04 0.47 0.44 0.08
2012-11-04 0.49 0.48 0.03
2012-11-04 0.52 0.46 0.01
2012-11-04 0.50 0.47 0.03
2012-11-05 0.51 0.46 0.02
2012-11-07 0.51 0.41 0.00
每个日期的输出只有一行。
编辑x2：修复错误
答案 0 :(得分：47)
这样的事情：
首先将数据帧重新采样为1D间隔。这取所有重复日期的值的平均值。使用fill_method选项填写缺少的日期值。接下来，将重新采样的帧传递到pd.rolling_mean，窗口为3，min_periods = 1：
pd.rolling_mean(df.resample("1D", fill_method="ffill"), window=3, min_periods=1)
favorable unfavorable other
enddate
2012-10-25 0.495000 0.485000 0.025000
2012-10-26 0.527500 0.442500 0.032500
2012-10-27 0.521667 0.451667 0.028333
2012-10-28 0.515833 0.450000 0.035833
2012-10-29 0.488333 0.476667 0.038333
2012-10-30 0.495000 0.470000 0.038333
2012-10-31 0.512500 0.460000 0.029167
2012-11-01 0.516667 0.456667 0.026667
2012-11-02 0.503333 0.463333 0.033333
2012-11-03 0.490000 0.463333 0.046667
2012-11-04 0.494000 0.456000 0.043333
2012-11-05 0.500667 0.452667 0.036667
2012-11-06 0.507333 0.456000 0.023333
2012-11-07 0.510000 0.443333 0.013333
更新：正如Ben在评论中指出的那样，with pandas 0.18.0 the syntax has changed。使用新语法，这将是：
df.resample("1d").sum().fillna(0).rolling(window=3, min_periods=1).mean()
答案 1 :(得分：34)
与此同时，增加了时间窗功能。请参阅以下链接：
https://github.com/pydata/pandas/pull/13513
In [1]: df = DataFrame({'B': range(5)})
In [2]: df.index = [Timestamp('20130101 09:00:00'),
...: Timestamp('20130101 09:00:02'),
...: Timestamp('20130101 09:00:03'),
...: Timestamp('20130101 09:00:05'),
...: Timestamp('20130101 09:00:06')]
In [3]: df
Out[3]:
B
2013-01-01 09:00:00 0
2013-01-01 09:00:02 1
2013-01-01 09:00:03 2
2013-01-01 09:00:05 3
2013-01-01 09:00:06 4
In [4]: df.rolling(2, min_periods=1).sum()
Out[4]:
B
2013-01-01 09:00:00 0.0
2013-01-01 09:00:02 1.0
2013-01-01 09:00:03 3.0
2013-01-01 09:00:05 5.0
2013-01-01 09:00:06 7.0
In [5]: df.rolling('2s', min_periods=1).sum()
Out[5]:
B
2013-01-01 09:00:00 0.0
2013-01-01 09:00:02 1.0
2013-01-01 09:00:03 3.0
2013-01-01 09:00:05 3.0
2013-01-01 09:00:06 7.0
答案 2 :(得分：33)
我只是有同样的问题，但数据点不规则。 Resample在这里不是一个真正的选择。所以我创建了自己的功能。也许它对其他人也有用：
from pandas import Series, DataFrame
import pandas as pd
from datetime import datetime, timedelta
import numpy as np
def rolling_mean(data, window, min_periods=1, center=False):
''' Function that computes a rolling mean
Parameters
----------
data : DataFrame or Series
If a DataFrame is passed, the rolling_mean is computed for all columns.
window : int or string
If int is passed, window is the number of observations used for calculating
the statistic, as defined by the function pd.rolling_mean()
If a string is passed, it must be a frequency string, e.g. '90S'. This is
internally converted into a DateOffset object, representing the window size.
min_periods : int
Minimum number of observations in window required to have a value.
Returns
-------
Series or DataFrame, if more than one column
'''
def f(x):
'''Function to apply that actually computes the rolling mean'''
if center == False:
dslice = col[x-pd.datetools.to_offset(window).delta+timedelta(0,0,1):x]
# adding a microsecond because when slicing with labels start and endpoint
# are inclusive
else:
dslice = col[x-pd.datetools.to_offset(window).delta/2+timedelta(0,0,1):
x+pd.datetools.to_offset(window).delta/2]
if dslice.size < min_periods:
return np.nan
else:
return dslice.mean()
data = DataFrame(data.copy())
dfout = DataFrame()
if isinstance(window, int):
dfout = pd.rolling_mean(data, window, min_periods=min_periods, center=center)
elif isinstance(window, basestring):
idx = Series(data.index.to_pydatetime(), index=data.index)
for colname, col in data.iterkv():
result = idx.apply(f)
result.name = colname
dfout = dfout.join(result, how='outer')
if dfout.columns.size == 1:
dfout = dfout.ix[:,0]
return dfout
# Example
idx = [datetime(2011, 2, 7, 0, 0),
datetime(2011, 2, 7, 0, 1),
datetime(2011, 2, 7, 0, 1, 30),
datetime(2011, 2, 7, 0, 2),
datetime(2011, 2, 7, 0, 4),
datetime(2011, 2, 7, 0, 5),
datetime(2011, 2, 7, 0, 5, 10),
datetime(2011, 2, 7, 0, 6),
datetime(2011, 2, 7, 0, 8),
datetime(2011, 2, 7, 0, 9)]
idx = pd.Index(idx)
vals = np.arange(len(idx)).astype(float)
s = Series(vals, index=idx)
rm = rolling_mean(s, window='2min')
答案 3 :(得分：7)
user2689410的代码正是我所需要的。提供我的版本（信用到user2689410），由于在DataFrame中为整行计算平均值，因此速度更快。
希望我的后缀约定是可读的：_s：string，_ i：int，_b：bool，_ser：Series和_df：DataFrame。如果找到多个后缀，则可以输入两种类型。
import pandas as pd
from datetime import datetime, timedelta
import numpy as np
def time_offset_rolling_mean_df_ser(data_df_ser, window_i_s, min_periods_i=1, center_b=False):
""" Function that computes a rolling mean
Credit goes to user2689410 at http://stackoverflow.com/questions/15771472/pandas-rolling-mean-by-time-interval
Parameters
----------
data_df_ser : DataFrame or Series
If a DataFrame is passed, the time_offset_rolling_mean_df_ser is computed for all columns.
window_i_s : int or string
If int is passed, window_i_s is the number of observations used for calculating
the statistic, as defined by the function pd.time_offset_rolling_mean_df_ser()
If a string is passed, it must be a frequency string, e.g. '90S'. This is
internally converted into a DateOffset object, representing the window_i_s size.
min_periods_i : int
Minimum number of observations in window_i_s required to have a value.
Returns
-------
Series or DataFrame, if more than one column
>>> idx = [
... datetime(2011, 2, 7, 0, 0),
... datetime(2011, 2, 7, 0, 1),
... datetime(2011, 2, 7, 0, 1, 30),
... datetime(2011, 2, 7, 0, 2),
... datetime(2011, 2, 7, 0, 4),
... datetime(2011, 2, 7, 0, 5),
... datetime(2011, 2, 7, 0, 5, 10),
... datetime(2011, 2, 7, 0, 6),
... datetime(2011, 2, 7, 0, 8),
... datetime(2011, 2, 7, 0, 9)]
>>> idx = pd.Index(idx)
>>> vals = np.arange(len(idx)).astype(float)
>>> ser = pd.Series(vals, index=idx)
>>> df = pd.DataFrame({'s1':ser, 's2':ser+1})
>>> time_offset_rolling_mean_df_ser(df, window_i_s='2min')
s1 s2
2011-02-07 00:00:00 0.0 1.0
2011-02-07 00:01:00 0.5 1.5
2011-02-07 00:01:30 1.0 2.0
2011-02-07 00:02:00 2.0 3.0
2011-02-07 00:04:00 4.0 5.0
2011-02-07 00:05:00 4.5 5.5
2011-02-07 00:05:10 5.0 6.0
2011-02-07 00:06:00 6.0 7.0
2011-02-07 00:08:00 8.0 9.0
2011-02-07 00:09:00 8.5 9.5
"""
def calculate_mean_at_ts(ts):
"""Function (closure) to apply that actually computes the rolling mean"""
if center_b == False:
dslice_df_ser = data_df_ser[
ts-pd.datetools.to_offset(window_i_s).delta+timedelta(0,0,1):
ts
]
# adding a microsecond because when slicing with labels start and endpoint
# are inclusive
else:
dslice_df_ser = data_df_ser[
ts-pd.datetools.to_offset(window_i_s).delta/2+timedelta(0,0,1):
ts+pd.datetools.to_offset(window_i_s).delta/2
]
if (isinstance(dslice_df_ser, pd.DataFrame) and dslice_df_ser.shape[0] < min_periods_i) or \
(isinstance(dslice_df_ser, pd.Series) and dslice_df_ser.size < min_periods_i):
return dslice_df_ser.mean()*np.nan # keeps number format and whether Series or DataFrame
else:
return dslice_df_ser.mean()
if isinstance(window_i_s, int):
mean_df_ser = pd.rolling_mean(data_df_ser, window=window_i_s, min_periods=min_periods_i, center=center_b)
elif isinstance(window_i_s, basestring):
idx_ser = pd.Series(data_df_ser.index.to_pydatetime(), index=data_df_ser.index)
mean_df_ser = idx_ser.apply(calculate_mean_at_ts)
return mean_df_ser
答案 4 :(得分：3)
这个例子似乎要求@ andyhayden的评论中建议的加权平均值。例如，10/25有两个民意调查，10/26和10/27分别有一个民意调查。如果您只是重新取样然后取平均值，这有效地为10/26和10/27的民意调查提供了两倍于10/25的民意调查。
为了给每个民意调查赋予相同的权重而不是每个天的权重相等，您可以执行以下操作。
>>> wt = df.resample('D',limit=5).count()
favorable unfavorable other
enddate
2012-10-25 2 2 2
2012-10-26 1 1 1
2012-10-27 1 1 1
>>> df2 = df.resample('D').mean()
favorable unfavorable other
enddate
2012-10-25 0.495 0.485 0.025
2012-10-26 0.560 0.400 0.040
2012-10-27 0.510 0.470 0.020
这为您提供了基于民意调查而不是基于日期的平均值的原始成分。和之前一样，民意调查的平均值为10/25，但10/25的权重也是存储的，并且是10/26或10/27的两倍，以反映10/25的两次民意调查。
>>> df3 = df2 * wt
>>> df3 = df3.rolling(3,min_periods=1).sum()
>>> wt3 = wt.rolling(3,min_periods=1).sum()
>>> df3 = df3 / wt3
favorable unfavorable other
enddate
2012-10-25 0.495000 0.485000 0.025000
2012-10-26 0.516667 0.456667 0.030000
2012-10-27 0.515000 0.460000 0.027500
2012-10-28 0.496667 0.465000 0.041667
2012-10-29 0.484000 0.478000 0.042000
2012-10-30 0.488000 0.474000 0.042000
2012-10-31 0.530000 0.450000 0.020000
2012-11-01 0.500000 0.465000 0.035000
2012-11-02 0.490000 0.470000 0.040000
2012-11-03 0.490000 0.465000 0.045000
2012-11-04 0.500000 0.448333 0.035000
2012-11-05 0.501429 0.450000 0.032857
2012-11-06 0.503333 0.450000 0.028333
2012-11-07 0.510000 0.435000 0.010000
请注意，10/27的滚动平均值现在为0.51500（轮询加权），而不是52.1667（日加权）。
另请注意，自版本0.18.0起，resample和rolling的API已发生变化。
答案 5 :(得分：3)
为了保持基本，我使用了一个循环和类似的东西让你开始（我的索引是日期时间）：
import pandas as pd
import datetime as dt
#populate your dataframe: "df"
#...
df[df.index<(df.index[0]+dt.timedelta(hours=1))] #gives you a slice. you can then take .sum() .mean(), whatever
然后您可以在该片上运行函数。您可以看到如何添加迭代器以使窗口的开始不是数据帧索引中的第一个值然后滚动窗口（例如，您可以使用＆gt;规则作为开始）。
请注意，对于SUPER大数据或非常小的增量，这可能效率较低，因为切片可能会变得更加费力（对于我来说，对于数十万行数据和几列来说非常有效，但是对于几周内的每小时窗口而言）
答案 6 :(得分：2)
我发现user2689410代码在我尝试使用window =＆＃39; 1M＆＃39;由于营业月的增量出现了这个错误：
AttributeError: 'MonthEnd' object has no attribute 'delta'
我添加了直接传递相对时间增量的选项，因此您可以为用户定义的时段执行类似的操作。
感谢指点，这是我的尝试 - 希望它有用。
def rolling_mean(data, window, min_periods=1, center=False):
""" Function that computes a rolling mean
Reference:
http://stackoverflow.com/questions/15771472/pandas-rolling-mean-by-time-interval
Parameters
----------
data : DataFrame or Series
If a DataFrame is passed, the rolling_mean is computed for all columns.
window : int, string, Timedelta or Relativedelta
int - number of observations used for calculating the statistic,
as defined by the function pd.rolling_mean()
string - must be a frequency string, e.g. '90S'. This is
internally converted into a DateOffset object, and then
Timedelta representing the window size.
Timedelta / Relativedelta - Can directly pass a timedeltas.
min_periods : int
Minimum number of observations in window required to have a value.
center : bool
Point around which to 'center' the slicing.
Returns
-------
Series or DataFrame, if more than one column
"""
def f(x, time_increment):
"""Function to apply that actually computes the rolling mean
:param x:
:return:
"""
if not center:
# adding a microsecond because when slicing with labels start
# and endpoint are inclusive
start_date = x - time_increment + timedelta(0, 0, 1)
end_date = x
else:
start_date = x - time_increment/2 + timedelta(0, 0, 1)
end_date = x + time_increment/2
# Select the date index from the
dslice = col[start_date:end_date]
if dslice.size < min_periods:
return np.nan
else:
return dslice.mean()
data = DataFrame(data.copy())
dfout = DataFrame()
if isinstance(window, int):
dfout = pd.rolling_mean(data, window, min_periods=min_periods, center=center)
elif isinstance(window, basestring):
time_delta = pd.datetools.to_offset(window).delta
idx = Series(data.index.to_pydatetime(), index=data.index)
for colname, col in data.iteritems():
result = idx.apply(lambda x: f(x, time_delta))
result.name = colname
dfout = dfout.join(result, how='outer')
elif isinstance(window, (timedelta, relativedelta)):
time_delta = window
idx = Series(data.index.to_pydatetime(), index=data.index)
for colname, col in data.iteritems():
result = idx.apply(lambda x: f(x, time_delta))
result.name = colname
dfout = dfout.join(result, how='outer')
if dfout.columns.size == 1:
dfout = dfout.ix[:, 0]
return dfout
以3天时间窗口计算平均值的例子：
from pandas import Series, DataFrame
import pandas as pd
from datetime import datetime, timedelta
import numpy as np
from dateutil.relativedelta import relativedelta
idx = [datetime(2011, 2, 7, 0, 0),
datetime(2011, 2, 7, 0, 1),
datetime(2011, 2, 8, 0, 1, 30),
datetime(2011, 2, 9, 0, 2),
datetime(2011, 2, 10, 0, 4),
datetime(2011, 2, 11, 0, 5),
datetime(2011, 2, 12, 0, 5, 10),
datetime(2011, 2, 12, 0, 6),
datetime(2011, 2, 13, 0, 8),
datetime(2011, 2, 14, 0, 9)]
idx = pd.Index(idx)
vals = np.arange(len(idx)).astype(float)
s = Series(vals, index=idx)
# Now try by passing the 3 days as a relative time delta directly.
rm = rolling_mean(s, window=relativedelta(days=3))
>>> rm
Out[2]:
2011-02-07 00:00:00 0.0
2011-02-07 00:01:00 0.5
2011-02-08 00:01:30 1.0
2011-02-09 00:02:00 1.5
2011-02-10 00:04:00 3.0
2011-02-11 00:05:00 4.0
2011-02-12 00:05:10 5.0
2011-02-12 00:06:00 5.5
2011-02-13 00:08:00 6.5
2011-02-14 00:09:00 7.5
Name: 0, dtype: float64
答案 7 :(得分：0)
检查您的索引确实是datetime，而不是str
会有所帮助：
data.index = pd.to_datetime(data['Index']).values
答案 8 :(得分：0)
可视化滚动平均值，看看它是否有意义。我不明白为什么在请求滚动平均值时使用 sum。
df=pd.read_csv('poll.csv',parse_dates=['enddate'],dtype={'favorable':np.float,'unfavorable':np.float,'other':np.float})
df.set_index('enddate')
df=df.fillna(0)
fig, axs = plt.subplots(figsize=(5,10))
df.plot(x='enddate', ax=axs)
plt.show()
df.rolling(window=3,min_periods=3).mean().plot()
plt.show()
print("The larger the window coefficient the smoother the line will appear")
print('The min_periods is the minimum number of observations in the window required to have a value')
df.rolling(window=6,min_periods=3).mean().plot()
plt.show()