Pandas / Python - 按时间段分组数据

时间:2017-01-14 19:50:08

标签: python pandas

我有一些财务数据,并希望只获得特定时间段(小时,天,月......)的最后一笔交易。

示例:

>>df
      time  price_BRL     qt              time_dt
1312001297      23.49   1.00  2011-07-30 04:48:17
1312049148      23.40   1.00  2011-07-30 18:05:48
1312121523      23.49   2.00  2011-07-31 14:12:03
1312121523      23.50   6.50  2011-07-31 14:12:03
1312177622      23.40   2.00  2011-08-01 05:47:02
1312206416      23.25   1.00  2011-08-01 13:46:56
1312637929      18.95   1.50  2011-08-06 13:38:49
1312637929      18.95   4.00  2011-08-06 13:38:49
1312817114       0.80   0.01  2011-08-08 15:25:14
1312818289       0.10   0.01  2011-08-08 15:44:49
1312819795       6.00   0.09  2011-08-08 16:09:55
1312847064      16.00   0.86  2011-08-08 23:44:24
1312849282      16.00   6.14  2011-08-09 00:21:22
1312898146      19.90   1.00  2011-08-09 13:55:46
1312915666       6.00   0.01  2011-08-09 18:47:46
1312934897      19.90   1.00  2011-08-10 00:08:17
>>filter_by_last_day(df)
      time  price_BRL     qt              time_dt
1312049148      23.40   1.00  2011-07-30 18:05:48
1312121523      23.50   6.50  2011-07-31 14:12:03
1312206416      23.25   1.00  2011-08-01 13:46:56
1312637929      18.95   4.00  2011-08-06 13:38:49
1312847064      16.00   0.86  2011-08-08 23:44:24
1312915666       6.00   0.01  2011-08-09 18:47:46
1312934897      19.90   1.00  2011-08-10 00:08:17

我正在考虑使用groupby()并获取当天的mean()(此解决方案也可用于我的问题,但不完全正确)但不知道如何选择df.groupby('time.day').last() {1}}

1 个答案:

答案 0 :(得分:3)

您可以groupby使用dt.date并按last汇总:

#if necessery convert to datetime
df.time_dt = pd.to_datetime(df.time_dt)

df = df.groupby(df.time_dt.dt.date).last().reset_index(drop=True)
print (df)
         time  price_BRL    qt             time_dt
0  1312049148      23.40  1.00 2011-07-30 18:05:48
1  1312121523      23.50  6.50 2011-07-31 14:12:03
2  1312206416      23.25  1.00 2011-08-01 13:46:56
3  1312637929      18.95  4.00 2011-08-06 13:38:49
4  1312847064      16.00  0.86 2011-08-08 23:44:24
5  1312915666       6.00  0.01 2011-08-09 18:47:46
6  1312934897      19.90  1.00 2011-08-10 00:08:17

感谢您MaxU寻求其他解决方案 - 为返回as_index=False添加参数DataFrame

df = df.groupby(df.time_dt.dt.date, as_index=False).last()
print (df)
         time  price_BRL    qt             time_dt
0  1312049148      23.40  1.00 2011-07-30 18:05:48
1  1312121523      23.50  6.50 2011-07-31 14:12:03
2  1312206416      23.25  1.00 2011-08-01 13:46:56
3  1312637929      18.95  4.00 2011-08-06 13:38:49
4  1312847064      16.00  0.86 2011-08-08 23:44:24
5  1312915666       6.00  0.01 2011-08-09 18:47:46
6  1312934897      19.90  1.00 2011-08-10 00:08:17

使用resample的解决方案,但必须按dropna删除NaN行:

df = df.resample('d', on='time_dt').last().dropna(how='all').reset_index(drop=True)
#cast column time to int
df.time = df.time.astype(int)
print (df)
         time  price_BRL    qt             time_dt
0  1312049148      23.40  1.00 2011-07-30 18:05:48
1  1312121523      23.50  6.50 2011-07-31 14:12:03
2  1312206416      23.25  1.00 2011-08-01 13:46:56
3  1312637929      18.95  4.00 2011-08-06 13:38:49
4  1312847064      16.00  0.86 2011-08-08 23:44:24
5  1312915666       6.00  0.01 2011-08-09 18:47:46
6  1312934897      19.90  1.00 2011-08-10 00:08:17

---

您还可以使用dt.month

df = df.groupby(df.time_dt.dt.month).last().reset_index(drop=True)
print (df)
         time  price_BRL   qt             time_dt
0  1312121523       23.5  6.5 2011-07-31 14:12:03
1  1312934897       19.9  1.0 2011-08-10 00:08:17

使用hours有点复杂,如果groupbydate需要hours,则解决方案将替换为minutesseconds 0之前的astype

hours = df.time_dt.values.astype('<M8[h]')
print (hours)
['2011-07-30T04' '2011-07-30T18' '2011-07-31T14' '2011-07-31T14'
 '2011-08-01T05' '2011-08-01T13' '2011-08-06T13' '2011-08-06T13'
 '2011-08-08T15' '2011-08-08T15' '2011-08-08T16' '2011-08-08T23'
 '2011-08-09T00' '2011-08-09T13' '2011-08-09T18' '2011-08-10T00']

df = df.groupby(hours).last().reset_index(drop=True)
print (df)
          time  price_BRL    qt             time_dt
0   1312001297      23.49  1.00 2011-07-30 04:48:17
1   1312049148      23.40  1.00 2011-07-30 18:05:48
2   1312121523      23.50  6.50 2011-07-31 14:12:03
3   1312177622      23.40  2.00 2011-08-01 05:47:02
4   1312206416      23.25  1.00 2011-08-01 13:46:56
5   1312637929      18.95  4.00 2011-08-06 13:38:49
6   1312818289       0.10  0.01 2011-08-08 15:44:49
7   1312819795       6.00  0.09 2011-08-08 16:09:55
8   1312847064      16.00  0.86 2011-08-08 23:44:24
9   1312849282      16.00  6.14 2011-08-09 00:21:22
10  1312898146      19.90  1.00 2011-08-09 13:55:46
11  1312915666       6.00  0.01 2011-08-09 18:47:46
12  1312934897      19.90  1.00 2011-08-10 00:08:17