是什么引起了com.esotericsoftware.kryo.KryoException:java.lang.NullPointerException'?序列化Java异常时

时间:2018-08-29 00:06:49

标签: apache-spark serialization kryo

当我执行以下简单的Spark代码时:

val trial = Try {
  require(
    requirement = false,
    "error!"
  )
}
val ee = trial.failed.get

sparkContext.parallelize(Seq(ee))
  .collect()

我遇到以下错误:

Job aborted due to stage failure: Task 3 in stage 0.0 failed 4 times, most recent failure: Lost task 3.3 in stage 0.0 (TID 6, 10.0.9.145, executor 1): com.esotericsoftware.kryo.KryoException: java.lang.NullPointerException
Serialization trace:
suppressedExceptions (java.lang.IllegalArgumentException)
    at com.esotericsoftware.kryo.serializers.ObjectField.write(ObjectField.java:101)
    at com.esotericsoftware.kryo.serializers.FieldSerializer.write(FieldSerializer.java:518)
    at com.esotericsoftware.kryo.Kryo.writeClassAndObject(Kryo.java:628)
    at com.esotericsoftware.kryo.serializers.DefaultArraySerializers$ObjectArraySerializer.write(DefaultArraySerializers.java:366)
    at com.esotericsoftware.kryo.serializers.DefaultArraySerializers$ObjectArraySerializer.write(DefaultArraySerializers.java:307)
    at com.esotericsoftware.kryo.Kryo.writeClassAndObject(Kryo.java:628)
    at org.apache.spark.serializer.KryoSerializerInstance.serialize(KryoSerializer.scala:315)
    at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:386)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.NullPointerException
    at java.util.Collections$UnmodifiableCollection.size(Collections.java:1030)
    at com.esotericsoftware.kryo.serializers.CollectionSerializer.write(CollectionSerializer.java:83)
    at com.esotericsoftware.kryo.serializers.CollectionSerializer.write(CollectionSerializer.java:40)
    at com.esotericsoftware.kryo.Kryo.writeObject(Kryo.java:552)
    at com.esotericsoftware.kryo.serializers.ObjectField.write(ObjectField.java:80)
    ... 10 more

显然,此错误是由我在调试中发现为emptyException的成员引起的。此外,我在互联网上找不到任何类似的错误。该错误的原因是什么,我应该怎么做才能避免它?

0 个答案:

没有答案