说我有一个列名列表,它们都存在于数据框中
Cols = ['A', 'B', 'C', 'D'],
我正在寻找一种获取表/数据框的快速方法
NA_counts min max
A 5 0 100
B 10 0 120
C 8 1 99
D 2 0 500
TIA
答案 0 :(得分:0)
您可以分别计算每个指标,然后像这样将所有指标并集:
nulls_cols = [sum(when(col(c).isNull(), lit(1)).otherwise(lit(0))).alias(c) for c in cols]
max_cols = [max(col(c)).alias(c) for c in cols]
min_cols = [min(col(c)).alias(c) for c in cols]
nulls_df = df.select(lit("NA_counts").alias("count"), *nulls_cols)
max_df = df.select(lit("Max").alias("count"), *max_cols)
min_df = df.select(lit("Min").alias("count"), *min_cols)
nulls_df.unionAll(max_df).unionAll(min_df).show()
输出示例:
+---------+---+---+----+----+
| count| A| B| C| D|
+---------+---+---+----+----+
|NA_counts| 1| 0| 3| 1|
| Max| 9| 5|Test|2017|
| Min| 1| 0|Test|2010|
+---------+---+---+----+----+