Question

以下内容：

val partials = rdd.mapPartitionsWithIndex((i, iter) => {
val (keys, values) = iter.toSeq.unzip  
val sums  = values.scanLeft(0)(_ + _) 
Iterator((keys.zip(sums.tail), sums.last))   
})

partials.collect

结果：

res12: Array[(Seq[(String, Int)], Int)] = Array((Stream((c01,1), ?),10), (Stream((c05,5), ?),18), (Stream((c08,8), ?),27))

我的问题是：什么是“？”代表什么？

Answer 1

流懒惰地评估其内容（可以是无限的）并记住被访问的部分。 ?之前的元素代表已经看过的项目，?代表尚未评估的Stream部分。

scala> val s = Stream.continually(1)  // Infinite stream of ones
s: scala.collection.immutable.Stream[Int] = Stream(1, ?)

scala> s  // Only the head of the stream is known
res3: scala.collection.immutable.Stream[Int] = Stream(1, ?)

scala> s.take(3).foreach(_ => ())  // Evaluate the first three elements

scala> s  // Now you the first three elements are known
res5: scala.collection.immutable.Stream[Int] = Stream(1, 1, 1, ?)

SPARK SCALA Stream？在输出中

1 个答案: