Question

假设我们有以下df：

+---+---------+---------+--------+-------+
|   |  2016   |  2017   |  2018  | 2019  |
+---+---------+---------+--------+-------+
| 0 | 26560.0 | 26810.0 | NaN    | NaN   |
| 1 |   570.0 | NaN     | 550.0  | 540.0 |
| 2 |  3770.0 | 3450.0  | 3210.0 | NaN   |
| 3 |  4320.0 | NaN     | NaN    | NaN   |
+---+---------+---------+--------+-------+

我想添加两个附加列“值”和“年”。在“值”列中，将包含最近一年的价值，而在“年”列中，应存在一个最新年份，其中没有缺失值：

+---+---------+---------+--------+-------+---------+------+
|   |  2016   |  2017   |  2018  | 2019  |  value  | year |
+---+---------+---------+--------+-------+---------+------+
| 0 | 26560.0 | 26810.0 | NaN    | NaN   | 26810.0 | 2017 |
| 1 |   570.0 | NaN     | 550.0  | 540.0 |   540.0 | 2019 |
| 2 |  3770.0 | 3450.0  | 3210.0 | NaN   |  3210.0 | 2018 |
| 3 |  4320.0 | NaN     | NaN    | NaN   |  4320.0 | 2016 |
+---+---------+---------+--------+-------+---------+------+

您能帮我解决这个问题吗？谢谢！

Answer 1

对新列使用DataFrame.assign，首先向前填充每行的缺失值，然后按位置选择最后一列，第二次按DataFrame.idxmax获取最后一个非缺失值，但必须通过索引更改列的顺序：

df1 = df.assign(value = df.ffill(axis=1).iloc[:, -1],
                year = df.notna().iloc[:, ::-1].idxmax(axis=1))
print (df1)
      2016     2017    2018   2019    value  year
0  26560.0  26810.0     NaN    NaN  26810.0  2017
1    570.0      NaN   550.0  540.0    540.0  2019
2   3770.0   3450.0  3210.0    NaN   3210.0  2018
3   4320.0      NaN     NaN    NaN   4320.0  2016

以上解决方案仅在至少存在非错值的情况下才有效，对于一般解决方案，如果缺少值，请使用numpy.where作为缺失值val：

print (df)
      2016     2017    2018   2019
0  26560.0  26810.0     NaN    NaN
1    570.0      NaN   550.0  540.0
2   3770.0   3450.0  3210.0    NaN
3      NaN      NaN     NaN    NaN

mask = df.notna()
df2 = df.assign(value = df.ffill(axis=1).iloc[:, -1],
               year = np.where(mask.any(axis=1), mask.iloc[:, ::-1].idxmax(axis=1), np.nan))
print (df2)
      2016     2017    2018   2019    value  year
0  26560.0  26810.0     NaN    NaN  26810.0  2017
1    570.0      NaN   550.0  540.0    540.0  2019
2   3770.0   3450.0  3210.0    NaN   3210.0  2018
3      NaN      NaN     NaN    NaN      NaN   NaN

如果某行仅包含缺失值，则使用DataFrame.stack和DataFrame.drop_duplicates的另一种想法也适用：

df2 = df.join(df.stack()
                .reset_index(name='value')
                .drop_duplicates('level_0', keep='last')
                .rename(columns={'level_1':'year'})
                .set_index('level_0')
                [['value','year']])
print (df2)
      2016     2017    2018   2019    value  year
0  26560.0  26810.0     NaN    NaN  26810.0  2017
1    570.0      NaN   550.0  540.0    540.0  2019
2   3770.0   3450.0  3210.0    NaN   3210.0  2018
3   4320.0      NaN     NaN    NaN   4320.0  2016

df2 = df.join(df.stack()
                .reset_index(name='value')
                .drop_duplicates('level_0', keep='last')
                .rename(columns={'level_1':'year'})
                .set_index('level_0')
                [['value','year']])
print (df2)
      2016     2017    2018   2019    value  year
0  26560.0  26810.0     NaN    NaN  26810.0  2017
1    570.0      NaN   550.0  540.0    540.0  2019
2   3770.0   3450.0  3210.0    NaN   3210.0  2018
3      NaN      NaN     NaN    NaN      NaN   NaN

Answer 2

使用pandas.Series.last_valid_index的其他方式：

def last_valid_value(series):
    ind = series.last_valid_index()
    return pd.Series([series[ind], ind])

df[["value", "year"]] = df.apply(last_valid_value, 1)
print(df)

输出：

      2016     2017    2018   2019    value  year
0  26560.0  26810.0     NaN    NaN  26810.0  2017
1    570.0      NaN   550.0  540.0    540.0  2019
2   3770.0   3450.0  3210.0    NaN   3210.0  2018
3   4320.0      NaN     NaN    NaN   4320.0  2016

Answer 3

您还可以将stack与groupby和last一起使用

s = df.stack()


df1 = df.assign(
    value=s.groupby(level=0).last(),
    year=s.reset_index(1).groupby(level=0)["level_1"].last(),
)

print(df1)

      2016     2017    2018   2019    value  year
1  26560.0  26810.0     NaN    NaN  26810.0  2017
2    570.0      NaN   550.0  540.0    540.0  2019
3   3770.0   3450.0  3210.0    NaN   3210.0  2018
4   4320.0      NaN     NaN    NaN   4320.0  2016

Answer 4

for i in df.index.values:
 for j in df.columns.values:
     if not pd.isna(df.loc[i,j] ):
         value=df.loc[i,j]
         year=j
 df.loc[i,"value"]=value
 df.loc[i,"year"]=year

熊猫基于缺少的值和列名称的新列

4 个答案: