Question

我正在使用kafka_2.10-0.10.0.1和scala_2.10.3。我想使用scala编写自定义Serializer和Deserializer。我尝试使用这些Serializer（来自CustomType）和Deserializer（获取CustomType）：

  class CustomTypeSerializer extends Serializer[CustomType] {
    private val gson: Gson = new Gson()

    override def configure(configs: util.Map[String, _], isKey: Boolean):
    Unit = {
      // nothing to do
    }

    override def serialize(topic: String, data: CustomType): Array[Byte] = {
      if (data == null)
        null
      else
        gson.toJson(data).getBytes
    }

    override def close(): Unit = {
      //nothing to do
    }
  }

  class CustomTypeDeserializer extends Deserializer[CustomType] {
    private val gson: Gson = new Gson()

    override def deserialize(topic: String, bytes: Array[Byte]): CustomType = {
      val offerJson = gson.toJson(bytes.toString)
      val psType: Type = new TypeToken[CustomType]() {}.getType()
      val ps: CustomType = gson.fromJson(offerJson, psType)
      ps
    }

    override def configure(configs: util.Map[String, _], isKey: Boolean):
    Unit = {
      // nothing to do
    }

    override def close(): Unit = {
      //nothing to do
    }
  }

但是，我收到了这个错误：

Exception in thread "main" org.apache.kafka.common.errors.SerializationException: Error deserializing key/value for partition topic_0_1-1 at offset 26
Caused by: com.google.gson.JsonSyntaxException: java.lang.IllegalStateException: Expected BEGIN_OBJECT but was BEGIN_ARRAY at line 1 column 2 path $
at com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$Adapter.read(ReflectiveTypeAdapterFactory.java:224)
at com.google.gson.Gson.fromJson(Gson.java:887)
at com.google.gson.Gson.fromJson(Gson.java:852)
at com.google.gson.Gson.fromJson(Gson.java:801)
at kafka.PSDeserializer.deserialize(PSDeserializer.scala:24)
at kafka.PSDeserializer.deserialize(PSDeserializer.scala:18)
at org.apache.kafka.clients.consumer.internals.Fetcher.parseRecord(Fetcher.java:627)
at org.apache.kafka.clients.consumer.internals.Fetcher.parseFetchedData(Fetcher.java:548)
at org.apache.kafka.clients.consumer.internals.Fetcher.fetchedRecords(Fetcher.java:354)
at org.apache.kafka.clients.consumer.KafkaConsumer.pollOnce(KafkaConsumer.java:1000)
at org.apache.kafka.clients.consumer.KafkaConsumer.poll(KafkaConsumer.java:938)

请帮助我

Answer 1

在下面找到用于案例类User，User（name：String，id：Int）的自定义序列化程序和反序列化程序。将代码中的User替换为您的案例类。它会起作用。

python manage.py runserver 8080

使用scala在kafka中创建自定义Serializer和Deserializer

1 个答案: