我是scala和spark的新手。在使用combineByKey时,有人可以帮助我理解以下错误。
val combineRdd = filterRdd.combineByKey( (value : Double) => (value ,1) , (acc : Double , value : Double) => ( (acc._1 + value ) , (acc._2 +1)) , (acc : Double, value : Double) => acc._1 + value._1 , acc._2 + value._2)
过滤器Rdd只是
scala> filterRdd.first
res6: (Int, Double) = (1,299.98)
错误是< console>:31:错误:值_1不是Double的成员
答案 0 :(得分:0)
这是正确的代码
val combineRdd = filterRdd.combineByKey( (value : Double) => (value ,1) , (acc : (Double, Int) , value : Double) => ( (acc._1 + value ) , (acc._2 +1)) , (acc : (Double , Int), value : (Double,Int ) ) => (acc._1 + value._1 , acc._2 + value._2))
combineRdd: org.apache.spark.rdd.RDD[(Int, (Double, Int))] = ShuffledRDD[3] at combineByKey at <console>:31