根据列的重复值折叠数据框并删除NaN值

时间:2019-05-25 07:17:44

标签: python pandas dataframe

我正在使用具有多个实验室值的患者数据库,即使每个实验室日期相同,每个实验室也会获得自己的行。我想根据每个患者的重复日期折叠行,以便每个日期有一行,其中包含当天所有实验室的结果。

我尝试了各种groupby()pd.merge()函数,但无济于事。

玩具示例:

import pandas as pd
import numpy as np
PID = [1, 1, 1, 2, 2, 2]
ALC = [200, np.nan, np.nan, 300, np.nan, np.nan]
WBC = [np.nan, 1000, np.nan, np.nan, 2000, np.nan]
per_neut = [np.nan, np.nan, 0.64, np.nan, np.nan, 0.77]
date = ['11/1/18', '11/2/18', '11/2/18', '1/11/04', 
        '1/11/04','1/11/04']

prac_dict = {'PID':PID, 'date':date, 'ALC':ALC, 'WBC':WBC,
             'per_neut':per_neut}
pract_df = pd.DataFrame(prac_dict)

这就是我所拥有的

print(pract_df)
   PID     date    ALC     WBC  per_neut
0    1  11/1/18  200.0     NaN       NaN
1    1  11/2/18    NaN  1000.0       NaN
2    1  11/2/18    NaN     NaN      0.64
3    2  1/11/04  300.0     NaN       NaN
4    2  1/11/04    NaN  2000.0       NaN
5    2  1/11/04    NaN     NaN      0.77

这就是我想要的:

   PID     date    ALC     WBC  per_neut
0    1  11/1/18  200.0     NaN       NaN
1    1  11/2/18    NaN  1000.0      0.64
2    2  1/11/04  300.0  2000.0      0.77

非常欢迎提出建议!

1 个答案:

答案 0 :(得分:0)

如果需要每列每组第一个不丢失的值,请使用GroupBy.first

df = pract_df.groupby(['PID','date'], as_index=False).first()
print (df)
   PID     date    ALC     WBC  per_neut
0    1  11/1/18  200.0     NaN       NaN
1    1  11/2/18    NaN  1000.0      0.64
2    2  1/11/04  300.0  2000.0      0.77

但是,如果需要每组重复的值,例如50列的最后一组中的ALC,则使用sum指定聚合函数,例如meanfirst }第二个值丢失:

PID = [1, 1, 1, 2, 2, 2]
ALC = [200, np.nan, np.nan, 300, np.nan, 50]
WBC = [np.nan, 1000, np.nan, np.nan, 2000, np.nan]
per_neut = [np.nan, np.nan, 0.64, np.nan, np.nan, 0.77]
date = ['11/1/18', '11/2/18', '11/2/18', '1/11/04', 
        '1/11/04','1/11/04']

prac_dict = {'PID':PID, 'date':date, 'ALC':ALC, 'WBC':WBC,
             'per_neut':per_neut}
pract_df = pd.DataFrame(prac_dict)
print (pract_df)
   PID     date    ALC     WBC  per_neut
0    1  11/1/18  200.0     NaN       NaN
1    1  11/2/18    NaN  1000.0       NaN
2    1  11/2/18    NaN     NaN      0.64
3    2  1/11/04  300.0     NaN       NaN
4    2  1/11/04    NaN  2000.0       NaN
5    2  1/11/04   50.0     NaN      0.77

df1 = pract_df.groupby(['PID','date'], as_index=False).sum(min_count=1)
print (df1)
   PID     date    ALC     WBC  per_neut
0    1  11/1/18  200.0     NaN       NaN
1    1  11/2/18    NaN  1000.0      0.64
2    2  1/11/04  350.0  2000.0      0.77

df2 = pract_df.groupby(['PID','date'], as_index=False).mean()
print (df2)
   PID     date    ALC     WBC  per_neut
0    1  11/1/18  200.0     NaN       NaN
1    1  11/2/18    NaN  1000.0      0.64
2    2  1/11/04  175.0  2000.0      0.77

df3 = pract_df.groupby(['PID','date'], as_index=False).first()
print (df3)
   PID     date    ALC     WBC  per_neut
0    1  11/1/18  200.0     NaN       NaN
1    1  11/2/18    NaN  1000.0      0.64
2    2  1/11/04  300.0  2000.0      0.77