CombineBy Key实现错误:value _1不是Double

时间:2016-08-15 07:10:30

标签: apache-spark

我是scala和spark的新手。在使用combineByKey时,有人可以帮助我理解以下错误。

val combineRdd = filterRdd.combineByKey( (value : Double) => (value  ,1)  , (acc : Double , value : Double) => ( (acc._1 + value ) , (acc._2 +1)) , (acc : Double, value : Double) => acc._1 + value._1 , acc._2 + value._2) 

过滤器Rdd只是

scala> filterRdd.first
res6: (Int, Double) = (1,299.98)

错误是< console>:31:错误:值_1不是Double的成员

1 个答案:

答案 0 :(得分:0)

这是正确的代码

val combineRdd = filterRdd.combineByKey( (value : Double) => (value ,1) , (acc : (Double, Int) , value : Double) => ( (acc._1 + value ) , (acc._2 +1)) , (acc : (Double , Int), value : (Double,Int ) ) => (acc._1 + value._1 , acc._2 + value._2)) combineRdd: org.apache.spark.rdd.RDD[(Int, (Double, Int))] = ShuffledRDD[3] at combineByKey at <console>:31