Question

我在与此类似的多索引DataFrame上运行groupby操作：

                                        0         1    ...
categories features subfeatures                    
cat1       feature1 subfeature1 -0.224487 -0.227524
                    subfeature2 -0.591399 -0.799228
           feature2 subfeature1  1.190110 -1.365895    ...
                    subfeature2  0.720956 -1.325562
cat2       feature1 subfeature1  1.856932       NaN
                    subfeature2 -1.354258 -0.740473
           feature2 subfeature1  0.234075 -1.362235    ...
                    subfeature2  0.013875  1.309564
cat3       feature1 subfeature1       NaN       NaN
                    subfeature2 -1.260408  1.559721    ...
           feature2 subfeature1  0.419246  0.084386
                    subfeature2  0.969270  1.493417

...                    ...               ...

可以使用以下代码生成它：

import pandas as pd, numpy as np
np.random.seed(seed=90)
results = np.random.randn(3,2,2,2)
results[2,0,0,:] = np.nan
results[1,0,0,1] = np.nan
results = results.reshape((-1,2))
index = pd.MultiIndex.from_product([["cat1", "cat2", "cat3"],
                                    ["feature1", "feature2"], 
                                    ["subfeature1", "subfeature2"]], 
                                   names=["categories", "features", "subfeatures"])
df = pd.DataFrame(results, index=index)

我正在尝试仅选择两个子特征数组之间的最大差异大于特定阈值的组，但我在groupby

时遇到问题

df.groupby(level=['categories','features'])

这给了我以下几组：

{('cat1', 'feature1'): [('cat1', 'feature1', 'subfeature1'),
  ('cat1', 'feature1', 'subfeature2')],
 ('cat1', 'feature2'): [('cat1', 'feature2', 'subfeature1'),
  ('cat1', 'feature2', 'subfeature2')],
 ('cat2', 'feature1'): [('cat2', 'feature1', 'subfeature1'),
  ('cat2', 'feature1', 'subfeature2')],
 ('cat2', 'feature2'): [('cat2', 'feature2', 'subfeature1'),
  ('cat2', 'feature2', 'subfeature2')],
 ('cat3', 'feature1'): [('cat3', 'feature1', 'subfeature1'),
  ('cat3', 'feature1', 'subfeature2')],
 ('cat3', 'feature2'): [('cat3', 'feature2', 'subfeature1'),
  ('cat3', 'feature2', 'subfeature2')]}

有没有办法进行分组，以便groupby函数忽略子特征级别？原因是我需要subfeature1和subfeature2一起，在不同的群组中他们没有价值。

理想情况下，我希望groupby返回类似这样的内容：

{('cat1', 'feature1'): [('cat1', 'feature1')],
 ('cat1', 'feature2'): [('cat1', 'feature2')],
 ('cat2', 'feature1'): [('cat2', 'feature1')],
 ('cat2', 'feature2'): [('cat2', 'feature2')],
 ('cat3', 'feature1'): [('cat3', 'feature1')],
 ('cat3', 'feature2'): [('cat3', 'feature2')],

我怎么能这样做？

Answer 1

In [20]: df.reset_index(level='subfeatures').groupby(level=['categories','features']).groups
Out[20]: 
{('cat1', 'feature1'): [('cat1', 'feature1'), ('cat1', 'feature1')],
 ('cat1', 'feature2'): [('cat1', 'feature2'), ('cat1', 'feature2')],
 ('cat2', 'feature1'): [('cat2', 'feature1'), ('cat2', 'feature1')],
 ('cat2', 'feature2'): [('cat2', 'feature2'), ('cat2', 'feature2')],
 ('cat3', 'feature1'): [('cat3', 'feature1'), ('cat3', 'feature1')],
 ('cat3', 'feature2'): [('cat3', 'feature2'), ('cat3', 'feature2')]}

Answer 2

在Jeff的帮助下，我设法找到了一个有效的解决方案。

def f(x):
    tmp = x.set_index('subfeatures')
    a = (tmp.xs('subfeature1')-tmp.xs('subfeature2')).abs().max()
    return a > 1

df.reset_index('subfeatures').groupby(level=['categories', 'features']).filter(f).set_index('subfeatures', append=True)

我基本上忽略subfeatures进行分组，然后暂时将其添加回过滤器函数中，但是这会丢失，所以我在过滤函数完成后最终确定它。

多指标Pandas groupby，忽略一个级别？

2 个答案: