Excel工作表中的数据存储如下:
Area | Product1 | Product2 | Product3
| sales|sales.Value| sales |sales.Value | sales |sales.Value
Location1 | 20 | 20000 | 25 | 10000 | 200 | 100
Location2 | 30 | 30000 | 3 | 12300 | 213 | 10
产品名称是给定月份的1000个左右区域中的每个区域的两个“销售数量”和“销售价值”的两个单元的合并。同样,过去5年每个月都有单独的文件。此外,新产品已在不同月份添加和删除。因此,不同的月份文件可能如下所示:
Area | Product1 | Product4 | Product3
论坛能否建议使用熊猫阅读此数据的最佳方式? 我不能使用索引,因为产品列每个月都不同
理想情况下,我想将上面的初始格式转换为:
Area | Product1.sales|Product1.sales.Value| Product2.sales |Product2.sales.Value |
Location1 | 20 | 20000 | 25 | 10000 |
Location2 | 30 | 30000 | 3 | 12300 |
import pandas as pd
xl_file = read_excel("file path", skiprow=2, sheetname=0)
/* since the first two rows are always blank */
0 1 2 3 4
0 NaN NaN NaN Auto loan NaN
1 Branch Code Branch Name Region No of accounts Portfolio Outstanding
2 3000 Name1 Central 0 0
3 3001 Name2 Central 0 0
我想将其转换为Auto loan.No of account
,Auto loan.Portfolio Outstanding
作为标题。
答案 0 :(得分:9)
假设您的DataFrame是df
:
import numpy as np
import pandas as pd
nan = np.nan
df = pd.DataFrame([
(nan, nan, nan, 'Auto loan', nan)
, ('Branch Code', 'Branch Name', 'Region', 'No of accounts'
, 'Portfolio Outstanding')
, (3000, 'Name1', 'Central', 0, 0)
, (3001, 'Name2', 'Central', 0, 0)
])
所以它看起来像这样:
0 1 2 3 4
0 NaN NaN NaN Auto loan NaN
1 Branch Code Branch Name Region No of accounts Portfolio Outstanding
2 3000 Name1 Central 0 0
3 3001 Name2 Central 0 0
然后首先向前填充前两行中的NaN(从而传播'Auto 贷款',例如)。
df.iloc[0:2] = df.iloc[0:2].fillna(method='ffill', axis=1)
接下来用空字符串填充剩余的NaN:
df.iloc[0:2] = df.iloc[0:2].fillna('')
现在将这两行与.
一起加入,并将其指定为列级别值:
df.columns = df.iloc[0:2].apply(lambda x: '.'.join([y for y in x if y]), axis=0)
最后,删除前两行:
df = df.iloc[2:]
这会产生
Branch Code Branch Name Region Auto loan.No of accounts \
2 3000 Name1 Central 0
3 3001 Name2 Central 0
Auto loan.Portfolio Outstanding
2 0
3 0
或者,您可以创建MultiIndex列而不是创建平面列索引:
import numpy as np
import pandas as pd
nan = np.nan
df = pd.DataFrame([
(nan, nan, nan, 'Auto loan', nan)
, ('Branch Code', 'Branch Name', 'Region', 'No of accounts'
, 'Portfolio Outstanding')
, (3000, 'Name1', 'Central', 0, 0)
, (3001, 'Name2', 'Central', 0, 0)
])
df.iloc[0:2] = df.iloc[0:2].fillna(method='ffill', axis=1)
df.iloc[0:2] = df.iloc[0:2].fillna('Area')
df.columns = pd.MultiIndex.from_tuples(
zip(*df.iloc[0:2].to_records(index=False).tolist()))
df = df.iloc[2:]
现在df
看起来像这样:
Area Auto loan
Branch Code Branch Name Region No of accounts Portfolio Outstanding
2 3000 Name1 Central 0 0
3 3001 Name2 Central 0 0
该列是MultiIndex:
In [275]: df.columns
Out[275]:
MultiIndex(levels=[[u'Area', u'Auto loan'], [u'Branch Code', u'Branch Name', u'No of accounts', u'Portfolio Outstanding', u'Region']],
labels=[[0, 0, 0, 1, 1], [0, 1, 4, 2, 3]])
该列有两个级别。第一级的值为[u'Area', u'Auto loan']
,第二级的值为[u'Branch Code', u'Branch Name', u'No of accounts', u'Portfolio Outstanding', u'Region']
。
然后,您可以通过指定两个级别的值来访问列:
print(df.loc[:, ('Area', 'Branch Name')])
# 2 Name1
# 3 Name2
# Name: (Area, Branch Name), dtype: object
print(df.loc[:, ('Auto loan', 'No of accounts')])
# 2 0
# 3 0
# Name: (Auto loan, No of accounts), dtype: object
使用MultiIndex的一个优点是您可以轻松选择具有特定级别值的所有列。例如,要选择与Auto loans
有关的子数据框,您可以使用:
In [279]: df.loc[:, 'Auto loan']
Out[279]:
No of accounts Portfolio Outstanding
2 0 0
3 0 0
有关从MultiIndex中选择行和列的详细信息,请参阅MultiIndexing Using Slicers。