我正在使用Spark并且有一个消息流,它们由消息启动分隔符分隔,
我想为每条消息创建一个单独的RDD,即。每次我们看到分隔符时。
这是否可以在Spark Streaming中使用?
修改
class RouteConsumer extends Actor with ActorHelper with Consumer {
def endpointUri = "rabbitmq://host:5672/xchange?queue=hl7"
def receive = {
case msg: CamelMessage =>
val m = msg.withBodyAs[String]
store(m.body)
}
}
val ssc = new StreamingContext(sparkConf, Seconds(3))
val dstream = ssc.actorStream[String](Props(new RouteConsumer()), "SparkReceiverActor")
val dStreamSplit = dstream.flatMap(_.split("MSH|^~\\&"))
由于