给定UnitOfWork
,例如
DataFrame
如何将每一行汇总成一个名为val df = sc.parallelize(Seq((1L, 0.1), (2L, 0.2), (3L, 0.3))).toDF("k","v")
df.show
+---+---+
| k| v|
+---+---+
| 1|0.1|
| 2|0.2|
| 3|0.3|
+---+---+
的新列,以便totals
dfTotals.show
答案 0 :(得分:1)
找到一个比原先想象的更简单的解决方案,
val totals = ($"k" + $"v")
val dfTotals = df.withColumn("totals", totals)
等等
dfTotals.show
+---+---+------+
| k| v|totals|
+---+---+------+
| 1|0.1| 1.1|
| 2|0.2| 2.2|
| 3|0.3| 3.3|
+---+---+------+
更新:另一种方法,虽然不是那么整洁,
df.select(df("k"), df("v"), df("k")+df("v"))