需要处理具有非唯一多索引的连接数据帧

时间:2018-04-12 01:51:15

标签: python python-3.x pandas dataframe

这有效:

import pandas as pd

raw_data = {
        'type_1': [1, 1],
        'id_1': ['2', '3'],
        'name_1': ['Alex', 'Amy']}
df_a = pd.DataFrame(raw_data, columns = ['type_1', 'id_1', 'name_1'])

raw_datab = {
        'type_2': [1, 1],
        'id_2': ['4', '5'],
        'name_2': ['Billy', 'Brian']}
    df_b = pd.DataFrame(raw_datab, columns = ['type_2', 'id_2', 'name_2'])

    dfs = [df_a.set_index(['type_1','id_1']),
           df_b.set_index(['type_2','id_2'])]

    df = pd.concat(dfs, axis=1)
    print (df)

打印:

     name_1 name_2
1 2    Amy    NaN
  3   Alex    NaN
  4    NaN  Billy
  5    NaN  Brian

如果我更改以下内容,则无效,因为raw_data中的多索引键是重复的:

     raw_data = {
        'type_1': [1, 1],    
        'id_1': ['2', '2'],   #  <-- changed from 3 to 2
        'name_1': ['Alex', 'Amy']}

以及以下内容:

raw_datab = {
        'type_2': [1, 1],
        'id_2': ['2', '5'], #  <-- changed from 4 to 2
        'name_2': ['Billy', 'Brian']}

因此,AlexAmyBilly都具有相同的多索引键[1,2],因此concat失败并显示:

  

无法处理非唯一的多索引!

但是重复数据是有效的,无论如何我需要连接它。这是我需要实现的结果(请注意,这应该是外连接,默认值):

     name_1  name_2
1 2    Amy    Billy
  2   Alex    Billy
  5    NaN    Brian

熊猫怎么可能这样做?

1 个答案:

答案 0 :(得分:5)

axis=1更改为axis=0(默认)

df = pd.concat(dfs)
df
Out[52]: 
            name_1 name_2
type_1 id_1              
1      2      Alex    NaN
       2       Amy    NaN
       4       NaN  Billy
       5       NaN  Brian

根据你的评论..

df_a.merge(df_b,left_on=['type_1','id_1'],right_on=['type_2','id_2'],how='outer').set_index(['type_2','id_2']).drop(['type_1','id_1'],1)
Out[80]: 
            name_1 name_2
type_2 id_2              
1      2      Alex  Billy
       2       Amy  Billy
       5       NaN  Brian