Dstream Spark流

时间:2018-10-29 13:54:07

标签: scala apache-spark spark-streaming

您好,我编写了以下代码

val receivedStream: DStream[PubsubSparkMessage] =
      PubsubSource.createStream(ssc, pubsubConfig, pubsubSubscriber)
    @volatile var receivedMessages: List[PubsubSparkMessage] = List() //scalastyle:ignore
    receivedStream.foreachRDD { rdd =>
      if (!rdd.isEmpty()) {
        receivedMessages = List(rdd.collect)
        receivedMessages
      }
    }

由于类型不匹配,我得到了一个错误; 我的问题是如何将List [Array [PubsubSparkMessage]]修改为List [PubsubSparkMessage]

1 个答案:

答案 0 :(得分:0)

类似的东西:

val list : List[Array[PubsubSparkMessage]] = List()

val result = list.flatMap(x => x)