Question

在将Spark 1.6应用程序移植到Spark 2.0.2的过程中，日志中存在此投诉：

com.esotericsoftware.kryo.KryoException: java.lang.IllegalArgumentException: Class is not registered: org.apache.spark.streaming.receiver.Receiver[]
Note: To register this class use: kryo.register(org.apache.spark.streaming.receiver.Receiver[].class);

这与Caused by: java.lang.ClassNotFoundException: org/apache/spark/streaming/receiver/Receiver[]/class：

失败

sparkConf.set("spark.kryo.classesToRegister", "org.apache.spark.streaming.receiver.Receiver[].class")

这与Caused by: java.lang.ClassNotFoundException: org/apache/spark/streaming/receiver/Receiver[]：

失败

sparkConf.set("spark.kryo.classesToRegister", "org.apache.spark.streaming.receiver.Receiver[]")

这与Class is not registered: org.apache.spark.streaming.receiver.Receiver[]：

失败

sparkConf.set("spark.kryo.classesToRegister", "org.apache.spark.streaming.receiver.Receiver")

这与Class is not registered: org.apache.spark.streaming.receiver.Receiver[]：

失败

sparkConf.registerKryoClasses(Array(
    classOf[org.apache.spark.streaming.receiver.Receiver[_]]
))

如何让这个课程注册？我已经能够用Kryo注册其他课程，但不能注册这个课程。

编辑：

在所有这些情况下，此设置已完成：

sparkConf.set("spark.kryo.registrationRequired", "true")
sparkConf.set("spark.serializer", classOf[KryoSerializer].getName)
GraphXUtils.registerKryoClasses(sparkConf)

Answer 1

找到一种适用于此的方法：Kryo serialization refuses to register class

简而言之，将classOf[org.apache.spark.streaming.receiver.Receiver[_]],更改为classOf[Array[org.apache.spark.streaming.receiver.Receiver[_]]],

如何在Kryo中注册Receiver []？

1 个答案: