我的pandas数据框如下所示，其中id和date是索引
id name date gross1 gross2 net1 net2 balance1 balance2
1 abc 01/01/2001 100 101 50 51 200 201
2 def 01/02/2001 201 202 40 41 300 3001
3 ghi 01/03/2001 300 303 99 98 1000 10001
我想对此进行转换，以便将数据转换为：
id date level parent category name value1 value1
1 01/01/2001 0 NaN gross abc 100 101
2 01/01/2001 1 1 net abc 50 51
3 01/01/2001 1 1 balance abc 200 201
4 01/02/2001 0 NaN gross def 201 201
5 01/02/2001 1 4 net def 40 41
6 01/02/2001 1 4 balance def 300 3001
7 01/03/2001 0 NaN gross ghi 300 303
8 01/03/2001 1 7 net ghi 99 98
9 01/03/2001 1 7 balance ghi 1000 10001
我尝试了旋转和取消堆叠...但是不能完全正确。关闭我来的是做下面的事情：
df_gross = df['name','gross1','gross2']
df_gross.columns = ['name', 'value1', 'value2']
df_gross['level']=0
df_gross['category']='gross'
df_net = df['name', 'net1','net2']
df_net.columns = ['name', 'value1', 'value2']
df_gross['level']=1
df_gross['category']='net'
df_balance = df['name', 'balance1','balance2']
df_balance.columns = ['name', 'value1', 'value2']
df_balance['level']=1
df_balance['category']='balance'
df = pandas.concat(df_gross, df_net, df_balance)
我遇到的问题是如何有效地生成新的id，并将父列设置为新生成的id。
我可以..在concat，reset_index之后，然后删除＆＃39; id＆＃39;然后将列设置为pandas生成的索引（应该是值1..n）。然后我想我做一个dataframe.apply来找到父母的日期＆＃39;和＆＃39;等级= 0＆＃39;并相应地设置父级。这会是最有效的方式吗？
答案 0 :(得分：3)
我能想到的最好方法是通过pandas重塑索引和名称，并用numpy重塑值。
首先，让我们在numpy中重塑数值：
arr = df.ix[:,'gross1':'balance2'].values.reshape(9,2)
array([[ 100, 101],
[ 50, 51],
[ 200, 201],
[ 201, 202],
[ 40, 41],
[ 300, 3001],
[ 300, 303],
[ 99, 98],
[ 1000, 10001]], dtype=int64)
现在让我们重塑pandas中的数据帧，使索引和列名更接近我们想要的内容：
df2 = df.set_index(['id','date','name']).stack().iloc[::2].reset_index().iloc[:,:-1]
id date name level_3
0 1 01/01/2001 abc gross1
1 1 01/01/2001 abc net1
2 1 01/01/2001 abc balance1
3 2 01/02/2001 def gross1
4 2 01/02/2001 def net1
5 2 01/02/2001 def balance1
6 3 01/03/2001 ghi gross1
7 3 01/03/2001 ghi net1
8 3 01/03/2001 ghi balance1
基本上90％，现在只是将它们结合起来：
df2[['value1','value2']] = pd.DataFrame(arr)
id date name level_3 value1 value2
0 1 01/01/2001 abc gross1 100 101
1 1 01/01/2001 abc net1 50 51
2 1 01/01/2001 abc balance1 200 201
3 2 01/02/2001 def gross1 201 202
4 2 01/02/2001 def net1 40 41
5 2 01/02/2001 def balance1 300 3001
6 3 01/03/2001 ghi gross1 300 303
7 3 01/03/2001 ghi net1 99 98
8 3 01/03/2001 ghi balance1 1000 10001
我不确定您打算如何使用关卡/父级colun，但是在这里可以设置它们：
df2['parent'] = df2.groupby('id').cumcount()
df2['parent_index'] = df2[ df2.parent == 0 ].index.to_series()
df2['parent_index'] = df2.parent_index.fillna(method='ffill')
df2['parent'] = np.where( df2.parent > 1, 1, df2.parent )
df2['parent_index'] = np.where( df2.parent == 0, np.nan, df2.parent_index )
id date name level_3 value1 value2 parent parent_index
0 1 01/01/2001 abc gross1 100 101 0 NaN
1 1 01/01/2001 abc net1 50 51 1 0
2 1 01/01/2001 abc balance1 200 201 1 0
3 2 01/02/2001 def gross1 201 202 0 NaN
4 2 01/02/2001 def net1 40 41 1 3
5 2 01/02/2001 def balance1 300 3001 1 3
6 3 01/03/2001 ghi gross1 300 303 0 NaN
7 3 01/03/2001 ghi net1 99 98 1 6
8 3 01/03/2001 ghi balance1 1000 10001 1 6
答案 1 :(得分：3)
这可以完全使用pandas完成。
import numpy as np
import pandas as pd
# assuming your dataframe is called `df`, first stack the dataframe
dfnew = df.set_index(['id', 'date','name']).stack().reset_index()
# split the category information into category and value level, then delete column level_3
dfnew[['category', 'valuelevel']] = dfnew.level_3.apply(
lambda x: pd.Series([x[:-1], x[-1]]))
del dfnew['level_3']
# reshape data to meet required format and reset_index
dfnew = dfnew.set_index(['id', 'date', 'name', 'category', 'valuelevel']).unstack(level=-1).reset_index()
# fix MultiIndex mess by flattening the column names,
# note: renaming id to parent because that is what it will end up being, new id will be the index.
dfnew.columns = ['parent', 'date', 'name', 'category', 'value1', 'value2']
# reorder the data frame according to parent_id & category ['gross', 'net', 'balance'],
# using a second data frame
# then get rid of the extra fields `index` & `catlevel`
cat_level = pd.DataFrame({'category': ['gross', 'net', 'balance'], 'catlevel': [0, 1, 2]})
dfnew = dfnew.merge(cat_level)
dfnew = dfnew.sort(['parent', 'catlevel']).reset_index()
del dfnew['index']
del dfnew['catlevel']
# generate the new row id from index
dfnew['id'] = dfnew.reset_index()['index'] + 1
# reset the parent column to point to the current parent id
dfnew['parent'] = dfnew.groupby('parent')['id'].transform(min)
# add new column level
dfnew['level'] = 1
# update the parent & level columns based on the mask parent == id
mask = dfnew.parent == dfnew.id
dfnew.level[mask] = 0
dfnew.parent[mask] = np.NaN
最终数据框如下所示：
parent date name category value1 value2 id level
0 NaN 01/01/2001 abc gross 100 101 1 0
1 1 01/01/2001 abc net 50 51 2 1
2 1 01/01/2001 abc balance 200 201 3 1
3 NaN 01/02/2001 def gross 201 202 4 0
4 4 01/02/2001 def net 40 41 5 1
5 4 01/02/2001 def balance 300 3001 6 1
6 NaN 01/03/2001 ghi gross 300 303 7 0
7 7 01/03/2001 ghi net 99 98 8 1
8 7 01/03/2001 ghi balance 1000 10001 9 1
列顺序不是您指定的，但形状和值是正确的。我不知道如何移动列，但我们可以使用正确的列顺序轻松创建新的数据框。
column_ordered = ['id', 'date', 'level', 'parent', 'category', 'name', 'value1', 'value2']
finaldf = pd.DataFrame()
for col in columns_ordered:
finaldf[col] = dfnew[col]