Question

我有一个顶级scala类，如下所示：

FinalOutput.scala：

class FinalOutput extends Serializable {

  @BeanProperty
  var userId: String = _

  @BeanProperty
  var tenantId: String = _

  @BeanProperty
  @SerializedName("type")
  var dataType: String = _

  @BeanProperty
  var data: FinalData = _

  @BeanProperty
  var userCreatedDate: String = _
}

FinalData.scala：

class FinalData extends Serializable {

  @BeanProperty
  var list1: ArrayBuffer[DataType1] = _

  @BeanProperty
  var list2: ArrayBuffer[DataType2] = _

  @BeanProperty
  var list3: ArrayBuffer[DataType3] = _

  @BeanProperty
  var list4: ArrayBuffer[DataType4] = _

  ....
  ....

  @BeanProperty
  var list15: ArrayBuffer[DataType15] = _

  @BeanProperty
  var userName: String = _
}

以及所有延伸DataType*

的BaseBean个类

我使用this将Scala对象序列化为json字符串。

ArrayBufferSerializer.scala

class ArrayBufferSerializer[T: ClassTag] extends JsonSerializer[ArrayBuffer[T]] {
  override def serialize(src: ArrayBuffer[T], typeOfSrc: Type, context: JsonSerializationContext): JsonElement = {
    context.serialize(src.toArray[Any])
  }
}

然后使用以下序列化为字符串：

val gson = new GsonBuilder().registerTypeAdapter(classOf[ArrayBuffer[FinalData]], new ArrayBufferSerializer[FinalData]()).serializeNulls.create
val data = gson.toJson(row)

现在我想将json字符串反序列化为FinalOutput对象，所以我创建了ArrayBufferDeSerializer这样的东西

class ArrayBufferDeSerializer[T: ClassTag] extends JsonDeserializer[ArrayBuffer[T]] {
  override def deserialize(json: JsonElement, typeOfT: Type, context: JsonDeserializationContext): ArrayBuffer[T] = {
    context.deserialize(json, typeOfT)
  }
}

然后调用下面的deserialzie：

val gson = new GsonBuilder().registerTypeAdapter(classOf[ArrayBuffer[FinalData]], new ArrayBufferSerializer[FinalData]()).serializeNulls.create
gson.fromJson(row, classOf[FinalLevelOneSmsOutput])

收到以下错误：

Exception in thread "main" org.apache.spark.SparkException: Job aborted due to stage failure: Task 0 in stage 2.0 failed 1 times, most recent failure: Lost task 0.0 in stage 2.0 (TID 3, localhost, executor driver): java.lang.StackOverflowError
    at com.google.gson.internal.bind.TypeAdapters$29.read(TypeAdapters.java:720)
    at com.google.gson.internal.bind.TypeAdapters$29.read(TypeAdapters.java:743)
    at com.google.gson.internal.bind.TypeAdapters$29.read(TypeAdapters.java:735)
    at com.google.gson.internal.bind.TypeAdapters$29.read(TypeAdapters.java:718)
    at com.google.gson.internal.Streams.parse(Streams.java:48)
    at com.google.gson.TreeTypeAdapter.read(TreeTypeAdapter.java:54)
    at com.google.gson.Gson.fromJson(Gson.java:861)
    at com.google.gson.Gson.fromJson(Gson.java:926)
    at com.google.gson.Gson$1.deserialize(Gson.java:131)
    at com.cv.util.ArrayBufferDeSerializer.deserialize(ArrayBufferDeSerializer.scala:15)
    at com.cv.util.ArrayBufferDeSerializer.deserialize(ArrayBufferDeSerializer.scala:13)
    at com.google.gson.TreeTypeAdapter.read(TreeTypeAdapter.java:58)

Answer 1

您的反序列化器除了将反序列化委托给具有相同参数的context（相同的json和相同类型）之外什么都不做，这将导致上下文再次调用反序列化器 - 创建无限循环和结果StackOverflowError。

解串器必须进行改进 - 因为我们序列化 ArrayBuffer进入＆＃34;简单＆＃34;数组，我们必须相应地反序列化它们。这是一种方法：

import com.google.gson.reflect.TypeToken class ArrayBufferDeSerializer[T: ClassTag] extends JsonDeserializer[ArrayBuffer[T]] { override def deserialize(json: JsonElement, typeOfT: Type, context: JsonDeserializationContext): ArrayBuffer[T] = { // since we've serialized ArrayBuffers by converting them to simple Arrays, we deserialize the // input as a simple Array first: val array: util.ArrayList[T] = context.deserialize(json, new TypeToken[Array[T]](){}.getType) // Then, we convert it back into an ArrayBuffer: import collection.JavaConverters._ ArrayBuffer[T](array.asScala: _*) } }

使用ArrayBuffer的Scala对象 - 使用Gson反序列化问题

1 个答案: