按列值过滤时间序列熊猫数据帧

时间:2020-04-29 13:47:27

标签: python pandas time-series filtering

此问题是此问题的附加内容:filter multi-indexed grouped pandas dataframe

我希望date之后的所有数据(按时间顺序)从第一个value开始为零。 (适用于每个id

示例输入数据:

id  timestamp   date        value
1   2001-01-01  2001-05-01  1
1   2001-10-01  2001-05-01  0
1   2001-10-02  2001-05-01  1
1   2001-10-03  2001-05-01  0
1   2001-10-04  2001-05-01  1

想要的输出数据示例:

id  timestamp   date        value
1   2001-10-02  2001-05-01  1
1   2001-10-03  2001-05-01  0
1   2001-10-04  2001-05-01  1

1 个答案:

答案 0 :(得分:1)

首先用Series.gt过滤另一列,然后创建GroupBy.cumsum,过滤更大的0,最后用DataFrame.reindex添加删除的值:

df['timestamp'] = pd.to_datetime(df['timestamp'])
df['date'] = pd.to_datetime(df['date'])
df = df.sort_values(['id','timestamp'])

m = df['timestamp'].gt(df['date'])
m1 = df[m].groupby('id')['value'].cumsum().gt(0).reindex(df.index, fill_value=False)
df = df[m1]
print (df)
   id  timestamp       date  value
2   1 2001-10-02 2001-05-01      1
3   1 2001-10-03 2001-05-01      0
4   1 2001-10-04 2001-05-01      1

另一个用Series.where替换列的想法:

df['timestamp'] = pd.to_datetime(df['timestamp'])
df['date'] = pd.to_datetime(df['date'])
df = df.sort_values(['id','timestamp'])

m = df['timestamp'].gt(df['date'])
m1 = df.assign(value = df['value'].where(m, 0)).groupby('id')['value'].cumsum().gt(0)
df = df[m1]
print (df)
   id  timestamp       date  value
2   1 2001-10-02 2001-05-01      1
3   1 2001-10-03 2001-05-01      0
4   1 2001-10-04 2001-05-01      1