我正在尝试使用以下形式的数据框执行任务（在生物信息学，TCGA数据中）：
df = pd.DataFrame({'ID':['TCGA-AB-0001','TCGA-AB-0001','TCGA-AB-0001','TCGA-AB-0001','TCGA-AB-0002','TCGA-AB-0002','TCGA-AB-0002','TCGA-AB-0002','TCGA-AB-0003','TCGA-AB-0002'],
'Reference':['HG19','HG18','HG19','GRCh37','HG18','HG19','GRCh37','HG19','GRCh37','GRCh37'],
'SampleType':['Tumor','Tumor','Normal','Normal','Tumor','Normal','Normal','Tumor','Tumor','Tumor']
})
看起来像：
ID Reference SampleType
0 TCGA-AB-0001 HG19 Tumor
1 TCGA-AB-0001 HG18 Tumor
2 TCGA-AB-0001 HG19 Normal
3 TCGA-AB-0001 GRCh37 Normal
4 TCGA-AB-0002 HG18 Tumor
5 TCGA-AB-0002 HG19 Normal
6 TCGA-AB-0002 GRCh37 Normal
7 TCGA-AB-0002 HG19 Tumor
8 TCGA-AB-0003 GRCh37 Tumor
9 TCGA-AB-0002 GRCh37 Tumor
如果它们具有相同的“参考”和不同的“SampleType”，我试图匹配行对。结果将是以下形式的新数据框：
TUMOR NORMAL
index ID Reference SampleType index ID Reference SampleType
0 TCGA-AB-0001 HG19 Tumor 2 TCGA-AB-0001 HG19 Normal
7 TCGA-AB-0002 HG19 Tumor 5 TCGA-AB-0002 HG19 Tumor
9 TCGA-AB-0002 GRCh37 Tumor 6 TCGA-AB-0002 GRCh37 Normal
现在我想丢弃重复的ID，但要这样做，优先级是根据列表[GRCh37，HG19，HG18]。因此，如果例如HG19和HG18都存在相同的ID，我将保留HG19。结果应如下所示：
TUMOR NORMAL
index ID Reference SampleType index ID Reference SampleType
0 TCGA-AB-0001 HG19 Tumor 2 TCGA-AB-0001 HG19 Normal
9 TCGA-AB-0002 GRCh37 Tumor 6 TCGA-AB-0002 GRCh37 Normal
有没有办法通过groupby或其他一些pandas函数来做到这一点？
谢谢！
答案 0 :(得分：2)
我仍然没有100％清楚所需的输出是什么。但这似乎是基于我的理解而做的伎俩。
import numpy as np
import pandas as pd
df = pd.DataFrame({'ID':['TCGA-AB-0001','TCGA-AB-0001','TCGA-AB-0001','TCGA-AB-0001','TCGA-AB-0002','TCGA-AB-0002','TCGA-AB-0002','TCGA-AB-0002','TCGA-AB-0003','TCGA-AB-0002', 'TCGA-AB-0001', 'TCGA-AB-0001'],
'Reference':['HG19','HG18','HG19','GRCh37','HG18','HG19','GRCh37','HG19','GRCh37','GRCh37', 'GRCh37', 'GRCh37'],
'SampleType':['Tumor','Tumor','Normal','Normal','Tumor','Normal','Normal','Tumor','Tumor','Tumor', 'Normal', 'Tumor']
})
这比原始示例略长，并且测试具有冗余候选行。
ID Reference SampleType
0 TCGA-AB-0001 HG19 Tumor
1 TCGA-AB-0001 HG18 Tumor
2 TCGA-AB-0001 HG19 Normal
3 TCGA-AB-0001 GRCh37 Normal
4 TCGA-AB-0002 HG18 Tumor
5 TCGA-AB-0002 HG19 Normal
6 TCGA-AB-0002 GRCh37 Normal
7 TCGA-AB-0002 HG19 Tumor
8 TCGA-AB-0003 GRCh37 Tumor
9 TCGA-AB-0002 GRCh37 Tumor
10 TCGA-AB-0001 GRCh37 Normal
11 TCGA-AB-0001 GRCh37 Tumor
现在我们创建一个可能具有＆＃34;冗余＆＃34;的临时df。行。
##
# Create the df with sort and first level filtering
##
df_2 = df.groupby(['ID','Reference']).filter(lambda x:set(x.SampleType)=={'Tumor','Normal'}).drop_duplicates(['ID', 'Reference', 'SampleType']).sort(['ID','Reference', 'SampleType'])
# By dropping dups and sorting, the SampleType column must alternate: Normal, Tumor, Normal...
# Break into two pieces for horizontal concat
left = df_2.iloc[np.arange(0,df_2.shape[0], 2)]
right = df_2.iloc[np.arange(1, df_2.shape[0], 2)]
# Reindex by ID so that pd.concat can properly match rows
left['old_index'] = left.index.values
left.index = left['ID']
right['old_index'] = right.index.values
right.index = right['ID']
right.columns = [c + '_2' for c in right.columns] # Rename right side columns so we can groupby(['ID'])
# Horizontal concat
temp = pd.concat([left, right], axis=1) # with possible duplicates for each unique (ID, Reference) tuple
temp.index = np.arange(temp.shape[0])
temp
ID Reference SampleType old_index ID_2 Reference_2 \
0 TCGA-AB-0001 GRCh37 Normal 3 TCGA-AB-0001 GRCh37
1 TCGA-AB-0001 HG19 Normal 2 TCGA-AB-0001 HG19
2 TCGA-AB-0002 GRCh37 Normal 6 TCGA-AB-0002 GRCh37
3 TCGA-AB-0002 HG19 Normal 5 TCGA-AB-0002 HG19
SampleType_2 old_index_2
0 Tumor 11
1 Tumor 0
2 Tumor 9
3 Tumor 7
如果我理解正确，我们只想为每个ID保留一行，按priority = ['GRCh37', 'HG19', 'HG18']的顺序选择
##
# Second level of filtering using priority list
##
priority = ['GRCh37', 'HG19', 'HG18']
g = temp.groupby(['ID'])
def filter_2(grp, priority = ['GRCh37', 'HG19', 'HG18']):
pos = np.argsort(grp['Reference'], priority).iloc[0]
idx = grp.index[pos]
return grp.loc[idx, :]
final = temp.groupby(['ID']).apply(filter_2)
final.index = np.arange(final.shape[0])
这使我理解了最终所需的输出。注意：这与原始示例不同，因为我扩展了输入df。
final
ID Reference SampleType old_index ID_2 Reference_2 \
0 TCGA-AB-0001 GRCh37 Normal 3 TCGA-AB-0001 GRCh37
1 TCGA-AB-0002 GRCh37 Normal 6 TCGA-AB-0002 GRCh37
SampleType_2 old_index_2
0 Tumor 11
1 Tumor 9
答案 1 :(得分：1)
为了创建新的数据框，您可以使用pandas conditionnal slicing :(在您的问题中，您在索引为5的行上的数据框NORMAL上出错了，SampleType应为{{ 1}}不Normal）
Tumor或者如果您有机会获得除NORMAL = df[df['SampleType']=='Normal'].copy()
TUMOR = df[df['SampleType']=='Tumor'].copy()
和'normal'之外的任何其他内容，除了'tumor'之外，您还无法获得其他所有内容：
'normal'然后，为了删除重复项并保留特定值，您可以创建另一个列，该列保留相同的信息，但由整数组成（比字符串列表更容易排序）：
NORMAL = df[df['SampleType']=='Normal']
TUMOR = df[~df['SampleType']=='Normal']
当然，您可以在拆分数据帧df之前执行此操作（然后您只在一个数据帧而不是两个数据帧上执行此操作）。完成本专栏：
NORMAL['Whatever'] = 0
TUMOR['Whatever'] = 0
然后按此新列排序，删除重复项，仅保留第一行：
NORMAL.ix[NORMAL['Reference'] == 'HG19','Whatever'] = 1
TUMOR.ix[TUMOR['Reference'] == 'HG19','Whatever'] = 1
NORMAL.ix[NORMAL['Reference'] == 'HG18','Whatever'] = 2
TUMOR.ix[TUMOR['Reference'] == 'HG18','Whatever'] = 2
为了获得预期的输出，删除临时列，并按索引求助：
NORMAL.sort_values(by = 'Whatever', inplace = True)
NORMAL.drop_duplicates(subset = 'ID',inplace = True)
TUMOR.sort_values(by = 'Whatever', inplace = True)
TUMOR.drop_duplicates(subset = 'ID',inplace = True)
输出：
NORMAL.drop('Whatever',1,inplace = True)
NORMAL.sort_index(inplace = True)
TUMOR.drop('Whatever',1,inplace = True)
TUMOR.sort_index(inplace = True)