如何使用删除NA值的选项来融合Pandas中的数据帧

Question

我有一个像这样的Pandas Dataframe：

df = pd.DataFrame({"VAR1":["V1","V2","V2","V3","V4","V4","V5"], "VAR2":["C1","C1","C1","C2","C2","C2","C3"], "VAR3":["S1","S2","S3","S4","","",""], "VAR4":["","S3","S4","S5","S6","",""], "VAR5":["","S7","","","","","S3"]})

df

我必须将其转换为看起来像这样的数据框

VAR1  VAR2  VALUE
V1     C1    S1
V2     C1    S2
V2     C1    S3
V2     C1    S7
V3     C2    S4
V3     C2    S5
V4     C2    S6
V5     C3    S3

也就是说，我想根据VAR, VAR4, VAR5

的映射将VAR1 and VAR2列合并为一列

Answer 1

您可以将melt与boolean indexing一起用于删除空值的行，然后在列中使用sort_values，将reset_index用于默认单调唯一index：< / p>

df = pd.melt(df, id_vars=['VAR1', 'VAR2'], value_name='VALUE').drop('variable', axis=1)
df = df[df.VALUE != ''].sort_values(['VAR1','VAR2']).reset_index(drop=True)
print (df)
  VAR1 VAR2 VALUE
0   V1   C1    S1
1   V2   C1    S2
2   V2   C1    S3
3   V2   C1    S3
4   V2   C1    S4
5   V2   C1    S7
6   V3   C2    S4
7   V3   C2    S5
8   V4   C2    S6
9   V5   C3    S3

如果需要drop_duplicates：

df = pd.melt(df, id_vars=['VAR1', 'VAR2'], value_name='VALUE').drop('variable', axis=1)
df = df[df.VALUE != ''].drop_duplicates().sort_values(['VAR1','VAR2']).reset_index(drop=True)
print (df)
  VAR1 VAR2 VALUE
0   V1   C1    S1
1   V2   C1    S2
2   V2   C1    S3
3   V2   C1    S4
4   V2   C1    S7
5   V3   C2    S4
6   V3   C2    S5
7   V4   C2    S6
8   V5   C3    S3

Answer 2

看起来你是在通过制作VAR1＆amp;组合而分组的第一行之后VAR2为关键。换句话说，你将从这些子集中删除所有重复项，但只是始终保持第一次出现。

要忽略空字符串的考虑，请将其替换为NaN并堆叠DF，以便完全删除它们。

grp_f = df.groupby(["VAR1", "VAR2"]).first()
grp_f.replace({"":np.nan}).stack().reset_index([0,1], name="VALUE").reset_index(drop=True)