python pandas过滤数据帧由另一个系列,多列

时间:2015-01-13 15:41:14

标签: python pandas

在获得最高交付数量的一系列日子之后,如何过滤掉那些日子的原始数据框?鉴于这两个:

most_liquid_contracts.head(20)
Out[32]: 
2007-04-26    706
2007-04-27    706
2007-04-29    706
2007-04-30    706
2007-05-01    706
2007-05-02    706
2007-05-03    706
2007-05-04    706
2007-05-06    706
2007-05-07    706
2007-05-08    706
2007-05-09    706
2007-05-10    706
2007-05-11    706
2007-05-13    706
2007-05-14    706
2007-05-15    706
2007-05-16    706
2007-05-17    706
2007-05-18    706
dtype: int64

df.head(20).to_string
Out[40]: 
<bound method DataFrame.to_string of                            
                              delivery  volume
2007-04-27 11:55:00+01:00       705       1
2007-04-27 13:46:00+01:00       705       1
2007-04-27 14:15:00+01:00       705       1
2007-04-27 14:33:00+01:00       705       1
2007-04-27 14:35:00+01:00       705       1
2007-04-27 17:05:00+01:00       705      16
2007-04-27 17:07:00+01:00       705       1
2007-04-27 17:12:00+01:00       705       1
2007-04-27 17:46:00+01:00       705       1
2007-04-27 18:25:00+01:00       705       2
2007-04-26 23:00:00+01:00       706      10
2007-04-26 23:01:00+01:00       706      12
2007-04-26 23:02:00+01:00       706       1
2007-04-26 23:05:00+01:00       706      21
2007-04-26 23:06:00+01:00       706      10
2007-04-26 23:07:00+01:00       706      19
2007-04-26 23:08:00+01:00       706       1
2007-04-26 23:13:00+01:00       706      10
2007-04-26 23:14:00+01:00       706      62
2007-04-26 23:15:00+01:00       706       3>

我试过了:

liquid = df[df.index.date==most_liquid_contracts.index & df['delivery']==most_liquid_contracts]

或者我是否需要合并?它看起来不太优雅,我也不确定......我试过了:

# ATTEMPT 1
most_liquid_contracts.index = pd.to_datetime(most_liquid_contracts.index, unit='d')
df['days'] = pd.to_datetime(df.index.date, unit='d')
mlc = most_liquid_contracts.to_frame(name='delivery')
mlc['days'] = mlc.index.date
data = pd.merge(mlc, df, on=['delivery', 'days'], left_index=True)

# ATTEMPT 2
liquid = pd.merge(mlc, df, on='delivery', how='inner', left_index=True)
# this gets me closer (ie. retains granularity), but somehow seems to be an outer join? it includes the union but not the intersection. this should be a subset of df, but instead has about x50 the rows, at around 195B. df originally has 4B

但我似乎无法保留原始“df”中所需的分钟级粒度。基本上,我只需要“df”仅适用于最具流动性的合约(来自most_liquid_contracts系列;例如,4月27日仅包含“706”标签合约,4月29日仅包含“706”标签合约)。然后是第二个完全相反的df:所有其他合约的df(即最具流动性)。

更新:了解更多信息 - enter image description here

2 个答案:

答案 0 :(得分:1)

棘手的部分是合并具有不同日期时间分辨率的索引的两个系列/数据帧。一旦你智能地组合它们,你就可以正常过滤。

# Make sure your series has a name
# Make sure the index is pure dates, not date 00:00:00
most_liquid_contracts.name = 'most'
most_liquid_conttracts.index = most_liquid_contracts.index.date

data = df
data['day'] = data.index.date
combined = data.join(most_liquid_contracts, on='day', how='left')

现在你可以做类似

的事情了
combined[combined.delivery == combined.most]

这将产生datadf)中的行,其中data.delivery等于当天most_liquid_contracts中的值。

答案 1 :(得分:0)

我假设我已正确理解您,并且most_liquid_contracts系列是包含某些整数N的N个最大交货的系列。您想要过滤df以仅包含交货数量足够高的天数以使名单。因此,您只需删除df中不大于most_liquid_contracts最小值的所有内容。

threshold = min(most_liquid_contracts)
filtered = df[df['delivery'] >= threshold]