Question

我在Pandas中有一个收集数据的数据框;

import pandas as pd
df = pd.DataFrame({'Group': ['A','A','A','A','A','A','A','B','B','B','B','B','B','B'], 'Subgroup': ['Blue', 'Blue','Blue','Red','Red','Red','Red','Blue','Blue','Blue','Blue','Red','Red','Red'],'Obs':[1,2,4,1,2,3,4,1,2,3,6,1,2,3]})

+-------+----------+-----+
| Group | Subgroup | Obs |
+-------+----------+-----+
| A     | Blue     |   1 |
| A     | Blue     |   2 |
| A     | Blue     |   4 |
| A     | Red      |   1 |
| A     | Red      |   2 |
| A     | Red      |   3 |
| A     | Red      |   4 |
| B     | Blue     |   1 |
| B     | Blue     |   2 |
| B     | Blue     |   3 |
| B     | Blue     |   6 |
| B     | Red      |   1 |
| B     | Red      |   2 |
| B     | Red      |   3 |
+-------+----------+-----+

观察（＆＃39; Obs＆＃39;）应该没有间隙编号，但你可以看到我们已经错过了＃39; A组中的蓝色3和B组中的蓝色4和5.期望的结果是所有“错过”的百分比。每组的观察结果（＆＃39; Obs＆＃39;），例如：

+-------+--------------------+--------+--------+
| Group | Total Observations | Missed |   %    |
+-------+--------------------+--------+--------+
| A     |                  8 |      1 | 12.5%  |
| B     |                  9 |      2 | 22.22% |
+-------+--------------------+--------+--------+

我尝试使用for循环和使用组（例如：

df.groupby(['Group','Subgroup']).sum()
print(groups.head)

）但我似乎无法以我尝试的任何方式工作。我是以错误的方式解决这个问题吗？

从another answer（大吼大叫到@Lie Ryan）我发现了一个寻找缺失元素的功能，但是我还不太了解如何实现这个功能;

def window(seq, n=2):
    "Returns a sliding window (of width n) over data from the iterable"
    "   s -> (s0,s1,...s[n-1]), (s1,s2,...,sn), ...                   "
    it = iter(seq)
    result = tuple(islice(it, n))
    if len(result) == n:
        yield result
    for elem in it:
        result = result[1:] + (elem,)
        yield result

def missing_elements(L):
    missing = chain.from_iterable(range(x + 1, y) for x, y in window(L) if (y - x) > 1)
    return list(missing)

任何人都可以给我指针是正确的方向吗？

Answer 1

很简单，你在这里需要groupby：

使用groupby + diff，找出每Group和SubGroup
df上的Group组，并计算上一步计算的列的size和sum
一些更简单的步骤（计算％）可以为您提供预期的输出。

f = [   # declare an aggfunc list in advance, we'll need it later
      ('Total Observations', 'size'), 
      ('Missed', 'sum')
]

g = df.groupby(['Group', 'Subgroup'])\
      .Obs.diff()\
      .sub(1)\
      .groupby(df.Group)\
      .agg(f)

g['Total Observations'] += g['Missed']
g['%'] = g['Missed'] / g['Total Observations'] * 100

g

       Total Observations  Missed          %
Group                                       
A                     8.0     1.0  12.500000
B                     9.0     2.0  22.222222

Answer 2

使用groupby，apply和assign的类似方法：

(
    df.groupby(['Group','Subgroup']).Obs
    .apply(lambda x: [x.max()-x.min()+1, x.max()-x.min()+1-len(x)])
    .apply(pd.Series)
    .groupby(level=0).sum()
    .assign(pct=lambda x: x[1]/x[0]*100)
    .set_axis(['Total Observations', 'Missed', '%'], axis=1, inplace=False)
)

Out[75]: 
       Total Observations  Missed          %
Group                                       
A                       8       1  12.500000
B                       9       2  22.222222

Answer 3

from collections import Counter

gs = ['Group', 'Subgroup']
old_tups = set(zip(*df.values.T))

missed = pd.Series(Counter(
    g for (g, s), d in df.groupby(gs)
    for o in range(d.Obs.min(), d.Obs.max() + 1)
    if (g, s, o) not in old_tups
), name='Missed')

hit = df.set_index(gs).Obs.count(level=0)
total = hit.add(missed).rename('Total')
ratio = missed.div(total).rename('%')

pd.concat([total, missed, ratio], axis=1).reset_index()

  Group  Total  Missed         %
0     A      8       1  0.125000
1     B      9       2  0.222222

计算子组

3 个答案: