通过映射(1到N)生成RDD失败

时间:2019-01-26 21:38:37

标签: apache-spark

这有效:

val rdd = sc.makeRDD((1 to 10))
val rdd2 = rdd.map(x => (1, 1,"2019-01-01", "2019-01-01",1,2,"XXXXXXXXXXXXXXXXXXXXXXXXXX"))

这不是:

val rdd = sc.makeRDD((1 to 10)).map((1, 1,"2019-01-01", "2019-01-01",1,2,"XXXXXXXXXXXXXXXXXXXXXXXXXX"))

得到的错误是:

notebook:1: error: type mismatch;
  found   : (Int, Int, String, String, Int, Int, String)
  required: Int => ?

这再次起作用,很好:

val rdd = sc.makeRDD((1 to  10)).map((_, 1,"2019-01-01", "2019-01-01",1,2,"XXXXXXXXXXXXXXXXXXXXXXXXXX"))

我在这里缺少一些要点。

1 个答案:

答案 0 :(得分:0)

缺少的一点是您必须传递一个函数,而(Int, Int, String, String, Int, Int, String)不是一个函数。

如果要生成常量

sc.makeRDD((1 to 10)).map(_ => (1, 1,"2019-01-01", "2019-01-01",1,2,"XXXXXXXXXXXXXXXXXXXXXXXXXX"))
相关问题