如何在Pandas中准确地将矩阵转换为柱状格式?

时间:2013-06-25 20:20:54

标签: pandas matrix

我有一个涉及500个流感序列的距离矩阵。我想将其转换为柱状格式,具有250,000个成对比较。是否有一项功能可以让我快速完成这项工作?

以下是我正在使用的数据集。索引是“Accession”列,我将数据表示为Pandas DataFrame。

    CY135678    CY142013    CY130339    CY130379    CY130460    CY135850    CY135930    CY143958    CY142889    CY141341    CY143073    CY142145    CY142817    CY142417    CY142985    CY136196    CY130412    CY135744    CY135326    CY135502
Accession                                                                               
CY135678     1.000000    0.959670    0.937148    0.932813    0.972692    0.951452    0.996966    0.998266    0.953619    0.993498    0.920628    0.956635    0.921936    0.956030    0.902904    0.968791    0.998700    0.952319    0.917642    0.922440
CY142013     0.959670    1.000000    0.939289    0.936253    0.963573    0.973981    0.956635    0.957936    0.974848    0.954033    0.923245    0.976149    0.924117    0.975620    0.913270    0.960104    0.958369    0.974848    0.923244    0.925926
CY130339     0.937148    0.939289    1.000000    0.975389    0.942783    0.938256    0.934114    0.935847    0.940415    0.935233    0.930222    0.939722    0.930659    0.939051    0.917098    0.938882    0.935847    0.939119    0.927612    0.927233
CY130379     0.932813    0.936253    0.975389    1.000000    0.935847    0.936960    0.929779    0.931946    0.939119    0.931347    0.923681    0.935820    0.924553    0.935133    0.915371    0.932813    0.931513    0.938687    0.925444    0.920697
CY130460     0.972692    0.963573    0.942783    0.935847    1.000000    0.955787    0.969658    0.970958    0.957087    0.966623    0.921936    0.961839    0.922809    0.961254    0.907239    0.991764    0.971391    0.957087    0.917642    0.920697
CY135850     0.951452    0.973981    0.938256    0.936960    0.955787    1.000000    0.947984    0.949718    0.993092    0.946891    0.922372    0.973114    0.923245    0.972573    0.909758    0.953619    0.950152    0.996546    0.916775    0.925054
CY135930     0.996966    0.956635    0.934114    0.929779    0.969658    0.947984    1.000000    0.996099    0.950152    0.991331    0.919320    0.953599    0.920628    0.952982    0.900737    0.965756    0.996532    0.948851    0.914608    0.919390
CY143958     0.998266    0.957936    0.935847    0.931946    0.970958    0.949718    0.996099    1.000000    0.951886    0.992631    0.919756    0.954900    0.921064    0.954288    0.901170    0.967057    0.997833    0.950585    0.916775    0.921569
CY142889     0.953619    0.974848    0.940415    0.939119    0.957087    0.993092    0.950152    0.951886    1.000000    0.949050    0.922372    0.973981    0.923245    0.973444    0.912349    0.954920    0.952319    0.993092    0.918075    0.925490
CY141341     0.993498    0.954033    0.935233    0.931347    0.966623    0.946891    0.991331    0.992631    0.949050    1.000000    0.919756    0.951431    0.921064    0.950805    0.896805    0.963589    0.993065    0.947755    0.915908    0.925054
CY143073     0.920628    0.923245    0.930222    0.923681    0.921936    0.922372    0.919320    0.919756    0.922372    0.919756    1.000000    0.921500    0.999128    0.917139    0.908853    0.917139    0.920192    0.923245    0.942433    0.938945
CY142145     0.956635    0.976149    0.939722    0.935820    0.961839    0.973114    0.953599    0.954900    0.973981    0.951431    0.921500    1.000000    0.921936    0.999565    0.911969    0.957936    0.955334    0.973981    0.918040    0.923747
CY142817     0.921936    0.924117    0.930659    0.924553    0.922809    0.923245    0.920628    0.921064    0.923245    0.921064    0.999128    0.921936    1.000000    0.917575    0.909725    0.918011    0.921500    0.924117    0.942870    0.939817
CY142417     0.956030    0.975620    0.939051    0.935133    0.961254    0.972573    0.952982    0.954288    0.973444    0.950805    0.917139    0.999565    0.917575    1.000000    0.911189    0.957336    0.954724    0.973444    0.917283    0.923312
CY142985     0.902904    0.913270    0.917098    0.915371    0.907239    0.909758    0.900737    0.901170    0.912349    0.896805    0.908853    0.911969    0.909725    0.911189    1.000000    0.902904    0.901170    0.911917    0.900737    0.905011
CY136196     0.968791    0.960104    0.938882    0.932813    0.991764    0.953619    0.965756    0.967057    0.954920    0.963589    0.917139    0.957936    0.918011    0.957336    0.902904    1.000000    0.967490    0.954920    0.913741    0.916340
CY130412     0.998700    0.958369    0.935847    0.931513    0.971391    0.950152    0.996532    0.997833    0.952319    0.993065    0.920192    0.955334    0.921500    0.954724    0.901170    0.967490    1.000000    0.951019    0.916342    0.921133
CY135744     0.952319    0.974848    0.939119    0.938687    0.957087    0.996546    0.948851    0.950585    0.993092    0.947755    0.923245    0.973981    0.924117    0.973444    0.911917    0.954920    0.951019    1.000000    0.918075    0.925926
CY135326     0.917642    0.923244    0.927612    0.925444    0.917642    0.916775    0.914608    0.916775    0.918075    0.915908    0.942433    0.918040    0.942870    0.917283    0.900737    0.913741    0.916342    0.918075    1.000000    0.949455
CY135502     0.922440    0.925926    0.927233    0.920697    0.920697    0.925054    0.919390    0.921569    0.925490    0.925054    0.938945    0.923747    0.939817    0.923312    0.905011    0.916340    0.921133    0.925926    0.949455    1.000000

应用affmat.unstack()后得到的输出如下:

       Accession
CY135678  CY135678     0.939085
          CY142013     0.959670
          CY130339     0.937148
          CY130379     0.932813
          CY130460     0.972692
          CY135850     0.951452
          CY135930     0.996966
          CY143958     0.998266
          CY142889     0.953619
          CY141341     0.993498
          CY143073     0.920628
          CY142145     0.956635
          CY142817     0.921936
          CY142417     0.956030
          CY142985     0.902904
...
CY135502  CY135850     0.925054
          CY135930     0.919390
          CY143958     0.921569
          CY142889     0.925490
          CY141341     0.925054
          CY143073     0.938945
          CY142145     0.923747
          CY142817     0.939817
          CY142417     0.923312
          CY142985     0.905011
          CY136196     0.916340
          CY130412     0.921133
          CY135744     0.925926
          CY135326     0.949455
          CY135502     0.939085
Length: 400, dtype: float64

从输出中可以看出,CY135678本身应该具有1.000000的身份,但在应用affmat.unstack()后变为0.939085。这种行为有解释吗?有没有什么方法可以让原始值正确堆叠?

2 个答案:

答案 0 :(得分:1)

也许您正在寻找unstack

In [29]: df
Out[29]: 
      a         b         c         d         e
a  0.453367  0.000969  0.199400  0.515258  0.610870
b  0.949461  0.002380  2.993674  1.357350  0.189058
c  0.117990  1.397985  0.093681  0.417855  0.686190
d  0.757732  4.975183  3.108177  0.019095  1.613240
e  0.022297  0.518517  0.006883  0.896779  0.485518

In [30]: df.unstack()
Out[30]: 
a  a    0.453367
   b    0.949461    
   c    0.117990
   d    0.757732
   e    0.022297
b  a    0.000969
   b    0.002380
   c    1.397985
   d    4.975183
   e    0.518517
c  a    0.199400
   b    2.993674
   c    0.093681
   d    3.108177
   e    0.006883
d  a    0.515258
   b    1.357350
   c    0.417855
   d    0.019095
   e    0.896779
e  a    0.610870
   b    0.189058
   c    0.686190
   d    1.613240
   e    0.485518

显示每次成对比较两次。要削减它,请考虑Joe Kington's comment on this answer中讨论的np.triu_indicies_from

答案 1 :(得分:0)

现在好了,不知怎的,我设法通过在导入数据时不将索引设置为'Accession'来解决问题。相反,我只在我真正需要索引为“加入”时才调用.set_index()函数。因此,当我想要堆叠数据时(而不是将其拆开),我最终做的是:

affmat_stacked = affmat.set_index('Accession').stack()

这似乎很好地解决了问题。再次感谢大家的所有提示!