Pandas DataFrame计算

时间:2015-04-17 23:38:11

标签: python python-2.7 pandas

我有一个相当复杂的数据框,如下所示:

df = pd.DataFrame({'0': {('Total Number of End Points', '0.01um', '0hr'): 12,
  ('Total Number of End Points', '0.1um', '0hr'): 8,
  ('Total Number of End Points', 'Control', '0hr'): 4,
  ('Total Number of End Points', '0.01um', '24hr'): 18,
  ('Total Number of End Points', '0.1um', '24hr'): 12,
  ('Total Number of End Points', 'Control', '24hr'): 6,
  ('Total Vessel Length', '0.01um', '0hr'): 12,
  ('Total Vessel Length', '0.1um', '0hr'): 8,
  ('Total Vessel Length', 'Control', '0hr'): 4,
  ('Total Vessel Length', '0.01um', '24hr'): 18,
  ('Total Vessel Length', '0.1um',  '24hr'): 12,
  ('Total Vessel Length', 'Control',  '24hr'): 6},
  '1': {('Total Number of End Points', '0.01um', '0hr'): 12,
  ('Total Number of End Points', '0.1um', '0hr'): 8,
  ('Total Number of End Points', 'Control', '0hr'): 4,
  ('Total Number of End Points', '0.01um', '24hr'): 18,
  ('Total Number of End Points', '0.1um', '24hr'): 12,
  ('Total Number of End Points', 'Control', '24hr'): 6,
  ('Total Vessel Length', '0.01um', '0hr'): 12,
  ('Total Vessel Length', '0.1um', '0hr'): 8,
  ('Total Vessel Length', 'Control', '0hr'): 4,
  ('Total Vessel Length', '0.01um', '24hr'): 18,
  ('Total Vessel Length', '0.1um',  '24hr'): 12,
  ('Total Vessel Length', 'Control',  '24hr'): 6},
  '2': {('Total Number of End Points', '0.01um', '0hr'): 12,
  ('Total Number of End Points', '0.1um', '0hr'): 8,
  ('Total Number of End Points', 'Control', '0hr'): 4,
  ('Total Number of End Points', '0.01um', '24hr'): 18,
  ('Total Number of End Points', '0.1um', '24hr'): 12,
  ('Total Number of End Points', 'Control', '24hr'): 6,
  ('Total Vessel Length', '0.01um', '0hr'): 12,
  ('Total Vessel Length', '0.1um', '0hr'): 8,
  ('Total Vessel Length', 'Control', '0hr'): 4,
  ('Total Vessel Length', '0.01um', '24hr'): 18,
  ('Total Vessel Length', '0.1um',  '24hr'): 12,
  ('Total Vessel Length', 'Control',  '24hr'): 6}})

print(df)
                                                 0   1   2
        Total Number of End Points 0.01um  0hr   12  12  12
                                           24hr  18  18  18
                                   0.1um   0hr    8   8   8
                                           24hr  12  12  12
                                   Control 0hr    4   4   4
                                           24hr   6   6   6
        Total Vessel Length        0.01um  0hr   12  12  12
                                           24hr  18  18  18
                                   0.1um   0hr    8   8   8
                                           24hr  12  12  12
                                   Control 0hr    4   4   4
                                           24hr   6   6   6

我试图将每个值除以相应控制级别中列的平均值。我试过以下但是没有用。

df2 = df.divide(df.xs('Control', level=1).mean(axis=1), axis='index')

我对python和pandas很陌生,所以我倾向于用MS Excel术语来思考这个问题。

如果是在Excel中,A1的公式('终点总数',' 0.01um',' 0hr',0)看起来像是:

=A1 / AVERAGE($A$5:$C$5)

B1('终点总数',' 0.01um',' 0hr',1)将是:

=B1 / AVERAGE($A$5:$C$5)

和A2('终点总数',' 0.01um',' 24小时',0)将是

=A1 / AVERAGE($A$6:$C$6)

此示例的预期结果是:

                                                 0  1  2
        Total Number of End Points 0.01um  0hr   3  3  3
                                           24hr  3  3  3
                                   0.1um   0hr   2  2  2
                                           24hr  2  2  2
                                   Control 0hr   1  1  1
                                           24hr  1  1  1
        Total Vessel Length        0.01um  0hr   3  3  3
                                           24hr  3  3  3
                                   0.1um   0hr   2  2  2
                                           24hr  2  2  2
                                   Control 0hr   1  1  1
                                           24hr  1  1  1

注意:实际数据中有许多索引和列。

2 个答案:

答案 0 :(得分:1)

Control值放在自己的列中会很有帮助。您可以使用unstack

执行此操作
df.index.names = ['field', 'type', 'time']
df2 = df.unstack(['type']).swaplevel(0, 1, axis=1)

# type                            0.01um 0.1um Control 0.01um 0.1um Control  \
#                                      0     0       0      1     1       1   
# field                      time                                             
# Total Number of End Points 0hr      12     8       4     12     8       4   
#                            24hr     18    12       6     18    12       6   
# Total Vessel Length        0hr      12     8       4     12     8       4   
#                            24hr     18    12       6     18    12       6   

# type                            0.01um 0.1um Control  
#                                      2     2       2  
# field                      time                       
# Total Number of End Points 0hr      12     8       4  
#                            24hr     18    12       6  
# Total Vessel Length        0hr      12     8       4  
#                            24hr     18    12       6  

现在找到每个Control的平均值:

ave = df2['Control'].mean(axis=1)
# field                       time
# Total Number of End Points  0hr     4
#                             24hr    6
# Total Vessel Length         0hr     4
#                             24hr    6
# dtype: float64

如您所料,您可以使用df2.divide来计算所需的结果。请务必使用axis=0告诉Pandas根据行索引匹配值(df2ave)。

result = df2.divide(ave, axis=0)
# type                            0.01um 0.1um Control 0.01um 0.1um Control  \
#                                      0     0       0      1     1       1   
# field                      time                                             
# Total Number of End Points 0hr       3     2       1      3     2       1   
#                            24hr      3     2       1      3     2       1   
# Total Vessel Length        0hr       3     2       1      3     2       1   
#                            24hr      3     2       1      3     2       1   

# type                            0.01um 0.1um Control  
#                                      2     2       2  
# field                      time                       
# Total Number of End Points 0hr       3     2       1  
#                            24hr      3     2       1  
# Total Vessel Length        0hr       3     2       1  
#                            24hr      3     2       1  

您所追求的基本上都是值。但是,如果您想重新排列DataFrame,使其与您发布的完全一致,那么:

result = result.stack(['type'])
result = result.reorder_levels(['field','type','time'], axis=0)
result = result.reindex(df.index)

产量

                                         0  1  2
field                      type    time         
Total Number of End Points 0.01um  0hr   3  3  3
                                   24hr  3  3  3
                           0.1um   0hr   2  2  2
                                   24hr  2  2  2
                           Control 0hr   1  1  1
                                   24hr  1  1  1
Total Vessel Length        0.01um  0hr   3  3  3
                                   24hr  3  3  3
                           0.1um   0hr   2  2  2
                                   24hr  2  2  2
                           Control 0hr   1  1  1
                                   24hr  1  1  1

全部放在一起:

df.index.names = ['field', 'type', 'time']
df2 = df.unstack(['type']).swaplevel(0, 1, axis=1)
ave = df2['Control'].mean(axis=1)
result = df2.divide(ave, axis=0)
result = result.stack(['type'])
result = result.reorder_levels(['field','type','time'], axis=0)
result = result.reindex(df.index)

答案 1 :(得分:0)

这里的问题是pandas被组织起来可以轻松地计算列,并且该问题需要从其他行中扣除一行的平均值。熊猫不是那样设计的。

但是,您可以使用转置.T轻松切换行和列,然后它可能更容易处理,实际上控制均值是一个班轮。

>>> df.T[(u'Total Vessel Length', u'Control', u'0hr')].mean()
4.0

此4.0来自原始数据中的两个4.0值:

>>> df.T[(u'Total Vessel Length', u'Control', u'0hr')]
a    4
b    4

此时看起来循环会解决问题。

未测试:

for primary in (u'Total Vessel Length',u'Total Number of End Points'):
     for um in (u'0.01um',u'0.1um'):
         for hours in (u'0hr',u'24hr'):
             df.T[(primary,um,hours)]=df.T[(primary,um,hours)]/df.T[(primary, u'Control', hours)].mean()

请注意,这并非划分非控制列,但很容易包含“控制”列。进入um循环。

更新这不起作用,不知道它是不是在修改数据框架。现在,我不确定为什么。

但是你可以通过在dict上调用pd.DataFrame来构造一个新的数据框 理解。

这似乎有效......

import pandas as pd

df = pd.DataFrame({'0': {('Total Number of End Points', '0.01um', '0hr'): 12,
  ('Total Number of End Points', '0.1um', '0hr'): 8,
  ('Total Number of End Points', 'Control', '0hr'): 4,
  ('Total Number of End Points', '0.01um', '24hr'): 18,
  ('Total Number of End Points', '0.1um', '24hr'): 12,
  ('Total Number of End Points', 'Control', '24hr'): 6,
  ('Total Vessel Length', '0.01um', '0hr'): 12,
  ('Total Vessel Length', '0.1um', '0hr'): 8,
  ('Total Vessel Length', 'Control', '0hr'): 4,
  ('Total Vessel Length', '0.01um', '24hr'): 18,
  ('Total Vessel Length', '0.1um',  '24hr'): 12,
  ('Total Vessel Length', 'Control',  '24hr'): 6},
  '1': {('Total Number of End Points', '0.01um', '0hr'): 12,
  ('Total Number of End Points', '0.1um', '0hr'): 8,
  ('Total Number of End Points', 'Control', '0hr'): 4,
  ('Total Number of End Points', '0.01um', '24hr'): 18,
  ('Total Number of End Points', '0.1um', '24hr'): 12,
  ('Total Number of End Points', 'Control', '24hr'): 6,
  ('Total Vessel Length', '0.01um', '0hr'): 12,
  ('Total Vessel Length', '0.1um', '0hr'): 8,
  ('Total Vessel Length', 'Control', '0hr'): 4,
  ('Total Vessel Length', '0.01um', '24hr'): 18,
  ('Total Vessel Length', '0.1um',  '24hr'): 12,
  ('Total Vessel Length', 'Control',  '24hr'): 6},
  '2': {('Total Number of End Points', '0.01um', '0hr'): 12,
  ('Total Number of End Points', '0.1um', '0hr'): 8,
  ('Total Number of End Points', 'Control', '0hr'): 4,
  ('Total Number of End Points', '0.01um', '24hr'): 18,
  ('Total Number of End Points', '0.1um', '24hr'): 12,
  ('Total Number of End Points', 'Control', '24hr'): 6,
  ('Total Vessel Length', '0.01um', '0hr'): 12,
  ('Total Vessel Length', '0.1um', '0hr'): 8,
  ('Total Vessel Length', 'Control', '0hr'): 4,
  ('Total Vessel Length', '0.01um', '24hr'): 18,
  ('Total Vessel Length', '0.1um',  '24hr'): 12,
  ('Total Vessel Length', 'Control',  '24hr'): 6}})

print df

df2 = pd.DataFrame({(primary,um,hours):df.T[(primary,um,hours)]/df.T[(primary,u'Control',hours)].mean() for primary in (u'Total Vessel Length',u'Total Number of End Points') for um in (u'0.01um',u'0.1um') for hours in (u'0hr',u'24hr')})

print df2.T

<强>输出

paul@home:~/SO$ python ./r.py 
                                              0   1   2
(Total Number of End Points, 0.01um, 0hr)    12  12  12
(Total Number of End Points, 0.01um, 24hr)   18  18  18
(Total Number of End Points, 0.1um, 0hr)      8   8   8
(Total Number of End Points, 0.1um, 24hr)    12  12  12
(Total Number of End Points, Control, 0hr)    4   4   4
(Total Number of End Points, Control, 24hr)   6   6   6
(Total Vessel Length, 0.01um, 0hr)           12  12  12
(Total Vessel Length, 0.01um, 24hr)          18  18  18
(Total Vessel Length, 0.1um, 0hr)             8   8   8
(Total Vessel Length, 0.1um, 24hr)           12  12  12
(Total Vessel Length, Control, 0hr)           4   4   4
(Total Vessel Length, Control, 24hr)          6   6   6

[12 rows x 3 columns]
                                            0  1  2
(Total Number of End Points, 0.01um, 0hr)   3  3  3
(Total Number of End Points, 0.01um, 24hr)  3  3  3
(Total Number of End Points, 0.1um, 0hr)    2  2  2
(Total Number of End Points, 0.1um, 24hr)   2  2  2
(Total Vessel Length, 0.01um, 0hr)          3  3  3
(Total Vessel Length, 0.01um, 24hr)         3  3  3
(Total Vessel Length, 0.1um, 0hr)           2  2  2
(Total Vessel Length, 0.1um, 24hr)          2  2  2

[8 rows x 3 columns]