尝试将数据帧与分类数据连接时发生意外错误

时间:2018-06-21 09:53:37

标签: python pandas concatenation categorical-data

我有两个看起来像这样的数据帧df1和df2:

#df1
                    counts    freqs
categories                 
automatic           13      0.40625
manual              19      0.59375

#df2

                    counts   freqs
categories                     
Straight Engine      18     0.5625
V engine             14     0.4375

任何人都可以解释为什么pd.concat([df1, df2], axis = 1)不给我这个东西吗

                    counts   freqs
categories                     
automatic               13  0.40625
manual                  19  0.59375 
Straight Engine         18  0.5625
V engine                14  0.4375

这是我尝试过的:

1-使用pd.concat()

我怀疑我构建这些数据框的方式可能是问题的根源。 这就是我最终得到这些特定数据帧的方式:

# imports
import pandas as pd
from pydataset import data # pip install pydataset to get datasets from R

# load data 
df_mtcars = data('mtcars')

# change dummyvariables to more describing variables:
df_mtcars['am'][df_mtcars['am'] == 0] = 'manual'
df_mtcars['am'][df_mtcars['am'] == 1] = 'automatic' 
df_mtcars['vs'][df_mtcars['vs'] == 0] = 'Straight Engine'
df_mtcars['vs'][df_mtcars['vs'] == 1] = 'V engine'

# describe categorical variables
df1 = pd.Categorical(df_mtcars['am']).describe()
df2 = pd.Categorical(df_mtcars['vs']).describe()

我了解“类别”是导致此处出现问题的原因,因为df_con = pd.concat([df1, df2], axis = 1)会引发此错误:

  

TypeError:追加时类别必须与现有类别匹配

但这让我感到困惑,没关系:

# code
df_con = pd.concat([df1, df2], axis = 1)

# output:
                 counts       freqs  counts   freqs
categories                                      
automatic          13.0     0.40625     NaN     NaN
manual             19.0     0.59375     NaN     NaN
Straight Engine     NaN         NaN    18.0  0.5625
V engine            NaN         NaN    14.0  0.4375

2-使用df.append()会引起与pd.concat()相同的错误

3-使用pd.merge()之类的作品,但我失去了索引:

# Code
df_merge = pd.merge(df1, df2, how = 'outer')

# Output
   counts    freqs
0      13  0.40625
1      19  0.59375
2      18  0.56250
3      14  0.43750

3-在转置的数据帧上使用pd.concat()

由于pd.concat()axis = 0一起工作,我想我会使用转置的数据帧到达那里。

# df1.T 
categories  automatic    manual
counts       13.00000  19.00000
freqs         0.40625   0.59375

# df2.T
categories  Straight Engine  V engine
counts              18.0000   14.0000
freqs                0.5625    0.4375

但仍然没有成功:

# code
df_con = pd.concat([df1.T, df2.T], axis = 1)

>>> TypeError: categories must match existing categories when appending

顺便说一句,我希望在这里是这样:

categories  automatic    manual Straight Engine  V engine
counts       13.00000  19.00000         18.0000   14.0000
freqs         0.40625   0.59375          0.5625    0.4375

尽管如此,axis = 0仍然可以工作:

# code  
df_con = pd.concat([df1.T, df2.T], axis = 0)

# Output
categories  automatic    manual  Straight Engine  V engine
counts       13.00000  19.00000              NaN       NaN
freqs         0.40625   0.59375              NaN       NaN
counts            NaN       NaN          18.0000   14.0000
freqs             NaN       NaN           0.5625    0.4375

但这与我要完成的目标相去甚远。

现在,我正在考虑可以从df1和df2中删除“类别”信息,但是我还没有找到解决方法。

感谢您提出其他建议!

1 个答案:

答案 0 :(得分:1)

尝试一下

pd.concat([df1.reset_index(),df2.reset_index()],ignore_index=True)

输出:

        categories  counts    freqs
0        automatic      13  0.40625
1           manual      19  0.59375
2  Straight Engine      18  0.56250
3         V engine      14  0.43750

要再次获得类别作为索引,请遵循此

pd.concat([df1.reset_index(),df2.reset_index()],ignore_index=True).set_index('categories')

输出:

                 counts    freqs
categories                      
automatic            13  0.40625
manual               19  0.59375
Straight Engine      18  0.56250
V engine             14  0.43750

有关更多详细信息,请遵循this docs