在一个火花流应用程序中,我收到的消息是,我用来作为RDD的项类传递的可序列化类之一,无法转换为接收器。我并没有试图把它变成一个接收器,但现在是:
15/04/18 18:30:22 ERROR Executor: Exception in task 1.0 in stage 0.0 (TID 1)
java.lang.ClassCastException: vehicles.Vehicle_Status cannot be cast to org.apache.spark.streaming.receiver.Receiver
at org.apache.spark.streaming.scheduler.ReceiverTracker$ReceiverLauncher$$anonfun$8.apply(ReceiverTracker.scala:295)
at org.apache.spark.streaming.scheduler.ReceiverTracker$ReceiverLauncher$$anonfun$8.apply(ReceiverTracker.scala:290)
at org.apache.spark.SparkContext$$anonfun$runJob$5.apply(SparkContext.scala:1497)
at org.apache.spark.SparkContext$$anonfun$runJob$5.apply(SparkContext.scala:1497)
at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:61)
at org.apache.spark.scheduler.Task.run(Task.scala:64)
at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:203)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
这里的代码可能会让我在那里:
val rmqReceiver = new RMQReceiver(qAddress, "vehicle-data")
val customReceiverStream = aContext.receiverStream(rmqReceiver)
val handler = (rdd: RDD[List[String]]) => {
this.handleStreamResult(rdd)
}
customReceiverStream.foreachRDD(handler)
def jsonToVehicleStatus(aRecord :String): Vehicle_Status ={
val ast = aRecord.parseJson
val aMap = ast.convertTo[Vehicle_Status]
return aMap
}
def handleStreamResult(rdd: RDD[List[String]]): Unit ={
rdd.foreach { list =>
val mapList = list.map(jsonToVehicleStatus)
val pMapList = sparkContext.parallelize(mapList)
pMapList.saveToCassandra("vehicle_data","vehicles",AllColumns)
println()
}
}
有没有人知道它为什么要把它投入接收器?