我是python和程序设计的新手,似乎找不到解决我问题的方法。我有一个从Excel工作表导入的数据框,其中包含15行物种及其数量和3列,它们是找到它们的位置。这是一个按站矩阵分类的物种:
A1 A2 A3
Species 1 1259 600 151
Species 2 912 1820 899
Species 3 1288 1491 631
Species 4 36 609 1946
Species 5 1639 819 1864
Species 6 1989 748 843
Species 7 688 271 1206
Species 8 1031 341 756
Species 9 1517 1164 138
Species 10 1290 669 811
Species 11 16 409 1686
Species 12 329 521 954
Species 13 1782 958 1727
Species 14 464 1804 1105
Species 15 1002 1483 109
我想为每列计算前10种(索引),它们的值,列中占总数的百分比,累积百分比,并在每个现有列之后插入新列并在一个数据帧中返回。
这是我要查找的结果(例如前两列):
Species A1 pct cum_pct Species A2 pct cum_pct
0 Species 6 1989 13 13 Species 2 1820 13 13
1 Species 13 1782 11 24 Species 14 1804 13 26
2 Species 5 1639 10 35 Species 3 1491 10 37
3 Species 9 1517 9 45 Species 15 1483 10 48
4 Species 10 1290 8 53 Species 9 1164 8 56
5 Species 3 1288 8 62 Species 13 958 6 63
6 Species 1 1259 8 70 Species 5 819 5 69
7 Species 8 1031 6 77 Species 6 748 5 75
8 Species 15 1002 6 83 Species 10 669 4 79
9 Species 2 912 5 89 Species 4 609 4 84
我设法通过计算每一列并创建新的数据帧,并使用concat最终使用以下代码将数据帧合并在一起来做到这一点:
df = pd.read_excel(r"") #local excel file
#extract first column and remove others
df = df.drop(df.columns[1:], axis=1)
# create column which has percentage for each element: divide value by total sum
df["pct"] = 100*(df.iloc[:, 0] /df.iloc[:, 0].sum())
#sort by value in Column 1 (0) return only top n (10) values
df = df.sort_values(by=df.columns[0], ascending=False).head(10)
# Create column with cumulative sum
df["cum_pct"] = df.pct.cumsum()
#make index as column and change name to Species
df = df.reset_index()
df = df.rename(index=str, columns={"index": "Species"})
# For column 2
df1 = pd.read_excel(r"") #local excel file
df1 = df1.drop(df1.columns[2:], axis=1)
df1 = df1.drop(df1.columns[0], axis=1)
# create column which has percentage for each element: divide value by total sum
df1["pct"] = 100*(df1.iloc[:, 0] /df1.iloc[:, 0].sum())
#sort by value in Column 1 (0) return only top n (10) values
df1 = df1.sort_values(by=df1.columns[0], ascending=False).head(10)
# Create column with cumulative sum
df1["cum_pct"] = df1.pct.cumsum()
# set index as first column
df1 = df1.reset_index()
df1 = df1.rename(index=str, columns={"index": "Species"})
# concatenate all dataframes
result = pd.concat([df, df1,], axis=1, join_axes=[df.index])
#convert numbers to int, exception = ignore
result = result.astype(int, errors="ignore")
print(result)
此代码有效,但是我的数据集要大得多,通常超过50列,因此我想知道是否有可能针对每一列进行迭代,从而得到与上述相同的数据帧。抱歉,长期阅读。
答案 0 :(得分:0)
使用for
循环,Series.nlargest
,DataFrame.assign
和lambda
函数来计算pct
和cum_pct
和pandas.concat
合并为最终输出帧:
frames = []
for col in df:
frames.append(df[col].nlargest(10).to_frame()
.assign(pct=lambda x: x[col] / df[col].sum(),
cum_pct=lambda x: x['pct'].cumsum())
.rename_axis('Species').reset_index())
df_new = pd.concat(frames, axis=1)
[出]
Species A1 pct cum_pct Species A2 pct cum_pct \
0 Species 6 1989 0.130495 0.130495 Species 2 1820 0.132779 0.132779
1 Species 13 1782 0.116914 0.247408 Species 14 1804 0.131612 0.264390
2 Species 5 1639 0.107532 0.354940 Species 3 1491 0.108777 0.373167
3 Species 9 1517 0.099528 0.454468 Species 15 1483 0.108193 0.481360
4 Species 10 1290 0.084635 0.539102 Species 9 1164 0.084920 0.566280
5 Species 3 1288 0.084503 0.623606 Species 13 958 0.069891 0.636171
6 Species 1 1259 0.082601 0.706207 Species 5 819 0.059750 0.695922
7 Species 8 1031 0.067642 0.773849 Species 6 748 0.054571 0.750492
8 Species 15 1002 0.065739 0.839588 Species 10 669 0.048807 0.799300
9 Species 2 912 0.059835 0.899423 Species 4 609 0.044430 0.843729
Species A3 pct cum_pct
0 Species 4 1946 0.131256 0.131256
1 Species 5 1864 0.125725 0.256981
2 Species 13 1727 0.116485 0.373466
3 Species 11 1686 0.113719 0.487185
4 Species 7 1206 0.081344 0.568528
5 Species 14 1105 0.074531 0.643059
6 Species 12 954 0.064346 0.707406
7 Species 2 899 0.060637 0.768043
8 Species 6 843 0.056860 0.824902
9 Species 10 811 0.054701 0.879603
如果有必要将计算的字段pct
和cum_pct
格式化为int
,则使用:
frames = []
for col in df:
frames.append(df[col].nlargest(10).to_frame()
.assign(pct=lambda x: x[col] / df[col].sum(),
cum_pct=lambda x: x['pct'].cumsum())
.assign(pct=lambda x: x['pct'].mul(100).astype(int),
cum_pct=lambda x: x['cum_pct'].mul(100).astype(int))
.rename_axis('Species').reset_index())
df_new = pd.concat(frames, axis=1)
[出]
Species A1 pct cum_pct Species A2 pct cum_pct Species \
0 Species 6 1989 13 13 Species 2 1820 13 13 Species 4
1 Species 13 1782 11 24 Species 14 1804 13 26 Species 5
2 Species 5 1639 10 35 Species 3 1491 10 37 Species 13
3 Species 9 1517 9 45 Species 15 1483 10 48 Species 11
4 Species 10 1290 8 53 Species 9 1164 8 56 Species 7
5 Species 3 1288 8 62 Species 13 958 6 63 Species 14
6 Species 1 1259 8 70 Species 5 819 5 69 Species 12
7 Species 8 1031 6 77 Species 6 748 5 75 Species 2
8 Species 15 1002 6 83 Species 10 669 4 79 Species 6
9 Species 2 912 5 89 Species 4 609 4 84 Species 10
A3 pct cum_pct
0 1946 13 13
1 1864 12 25
2 1727 11 37
3 1686 11 48
4 1206 8 56
5 1105 7 64
6 954 6 70
7 899 6 76
8 843 5 82
9 811 5 87