使用Spring KafkaListener处理JSON反序列化错误

时间：2019-02-28 17:25:02

标签： spring-kafka

我正在使用Spring KafkaListener来消耗来自Kafka主题的一些消息。查看日志，尝试将JSON消息反序列化到bean时，我注意到一个错误。

抛出的异常是这样：

Caused by: com.fasterxml.jackson.databind.exc.MismatchedInputException: Cannot deserialize instance of `java.util.ArrayList` out of VALUE_STRING token
 at [Source: (byte[])"{"id": 42, "userId": "", "type": "EMAIL", "status": "CREATE", "creationDate": "2019-02-26T11:13:44.000+0000", "lastModificationDate": "2019-02-26T11:13:44.000+0000", "title": "Your weekly Google Ads update: (Week of 02/11- 02/17)", "subject": "Your weekly Google Ads update: (Week of 02/11- 02/17)", "conversationOn": "", "mainKeyword": "Google Ads App", "keywords": ["Google Ads App", "People", "GBP", "AdWords App", "Google Ireland Ltd"], "actions": [], "emails""[truncated 8510 bytes]; line: 1, column: 752] (through reference chain: event.model.EmailTopicMessage["emails"]->feed.domain.email.Emails["messages"]->java.util.ArrayList[0]->feed.domain.email.Message["from"])
    at com.fasterxml.jackson.databind.exc.MismatchedInputException.from(MismatchedInputException.java:63) ~[jackson-databind-2.9.7.jar!/:2.9.7]
    at com.fasterxml.jackson.databind.DeserializationContext.reportInputMismatch(DeserializationContext.java:1343) ~[jackson-databind-2.9.7.jar!/:2.9.7]
    at com.fasterxml.jackson.databind.DeserializationContext.handleUnexpectedToken(DeserializationContext.java:1139) ~[jackson-databind-2.9.7.jar!/:2.9.7]
    at com.fasterxml.jackson.databind.DeserializationContext.handleUnexpectedToken(DeserializationContext.java:1093) ~[jackson-databind-2.9.7.jar!/:2.9.7]
    at com.fasterxml.jackson.databind.deser.std.CollectionDeserializer.handleNonArray(CollectionDeserializer.java:332) ~[jackson-databind-2.9.7.jar!/:2.9.7]
    at com.fasterxml.jackson.databind.deser.std.CollectionDeserializer.deserialize(CollectionDeserializer.java:265) ~[jackson-databind-2.9.7.jar!/:2.9.7]
    at com.fasterxml.jackson.databind.deser.std.CollectionDeserializer.deserialize(CollectionDeserializer.java:245) ~[jackson-databind-2.9.7.jar!/:2.9.7]
    at com.fasterxml.jackson.databind.deser.std.CollectionDeserializer.deserialize(CollectionDeserializer.java:27) ~[jackson-databind-2.9.7.jar!/:2.9.7]
    at com.fasterxml.jackson.databind.deser.impl.MethodProperty.deserializeAndSet(MethodProperty.java:127) ~[jackson-databind-2.9.7.jar!/:2.9.7]
    at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserializeFromObject(BeanDeserializer.java:369) ~[jackson-databind-2.9.7.jar!/:2.9.7]
    at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:159) ~[jackson-databind-2.9.7.jar!/:2.9.7]
    at com.fasterxml.jackson.databind.deser.std.CollectionDeserializer.deserialize(CollectionDeserializer.java:286) ~[jackson-databind-2.9.7.jar!/:2.9.7]
    at com.fasterxml.jackson.databind.deser.std.CollectionDeserializer.deserialize(CollectionDeserializer.java:245) ~[jackson-databind-2.9.7.jar!/:2.9.7]
    at com.fasterxml.jackson.databind.deser.std.CollectionDeserializer.deserialize(CollectionDeserializer.java:27) ~[jackson-databind-2.9.7.jar!/:2.9.7]
    at com.fasterxml.jackson.databind.deser.impl.MethodProperty.deserializeAndSet(MethodProperty.java:127) ~[jackson-databind-2.9.7.jar!/:2.9.7]
    at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserializeFromObject(BeanDeserializer.java:369) ~[jackson-databind-2.9.7.jar!/:2.9.7]
    at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:159) ~[jackson-databind-2.9.7.jar!/:2.9.7]
    at com.fasterxml.jackson.databind.deser.impl.MethodProperty.deserializeAndSet(MethodProperty.java:127) ~[jackson-databind-2.9.7.jar!/:2.9.7]
    at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserializeFromObject(BeanDeserializer.java:369) ~[jackson-databind-2.9.7.jar!/:2.9.7]
    at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:159) ~[jackson-databind-2.9.7.jar!/:2.9.7]
    at com.fasterxml.jackson.databind.ObjectReader._bindAndClose(ObjectReader.java:1611) ~[jackson-databind-2.9.7.jar!/:2.9.7]
    at com.fasterxml.jackson.databind.ObjectReader.readValue(ObjectReader.java:1234) ~[jackson-databind-2.9.7.jar!/:2.9.7]
    at org.springframework.kafka.support.serializer.JsonDeserializer.deserialize(JsonDeserializer.java:257) ~[spring-kafka-2.1.11.RELEASE.jar!/:2.1.11.RELEASE]
    at org.springframework.kafka.support.serializer.JsonDeserializer.deserialize(JsonDeserializer.java:233) ~[spring-kafka-2.1.11.RELEASE.jar!/:2.1.11.RELEASE]
    at org.apache.kafka.clients.consumer.internals.Fetcher.parseRecord(Fetcher.java:923) ~[kafka-clients-1.0.2.jar!/:na]
    at org.apache.kafka.clients.consumer.internals.Fetcher.access$2600(Fetcher.java:93) ~[kafka-clients-1.0.2.jar!/:na]
    at org.apache.kafka.clients.consumer.internals.Fetcher$PartitionRecords.fetchRecords(Fetcher.java:1100) ~[kafka-clients-1.0.2.jar!/:na]
    at org.apache.kafka.clients.consumer.internals.Fetcher$PartitionRecords.access$1200(Fetcher.java:949) ~[kafka-clients-1.0.2.jar!/:na]
    at org.apache.kafka.clients.consumer.internals.Fetcher.fetchRecords(Fetcher.java:570) ~[kafka-clients-1.0.2.jar!/:na]
    at org.apache.kafka.clients.consumer.internals.Fetcher.fetchedRecords(Fetcher.java:531) ~[kafka-clients-1.0.2.jar!/:na]
    at org.apache.kafka.clients.consumer.KafkaConsumer.pollOnce(KafkaConsumer.java:1154) ~[kafka-clients-1.0.2.jar!/:na]
    at org.apache.kafka.clients.consumer.KafkaConsumer.poll(KafkaConsumer.java:1111) ~[kafka-clients-1.0.2.jar!/:na]
    at org.springframework.kafka.listener.KafkaMessageListenerContainer$ListenerConsumer.run(KafkaMessageListenerContainer.java:712) ~[spring-kafka-2.1.11.RELEASE.jar!/:2.1.11.RELEASE]
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [na:1.8.0_171]
    at java.util.concurrent.FutureTask.run(FutureTask.java:266) [na:1.8.0_171]
    at java.lang.Thread.run(Thread.java:748) [na:1.8.0_171]

如何处理错误并继续下一条消息？

关于如何处理这些类型的错误，记录到文件或放入另一个队列进行分析调查和重新处理的最佳实践是什么？

任何示例代码都将受到高度赞赏。

1 个答案:

答案 0 :(得分：1)

您需要配置一个ErrorHandlingDeserializer2来包装该JSON，以便在反序列化期间更好地处理错误。

有关更多信息，请参阅文档：https://docs.spring.io/spring-kafka/docs/2.2.4.RELEASE/reference/#error-handling-deserializer

  

当反序列化器无法反序列化消息时，Spring无法处理该问题，因为它发生在poll()返回之前。为解决此问题，版本2.2引入了ErrorHandlingDeserializer2。该解串器将委托给实际的解串器（键或值）。如果委托未能反序列化记录内容，则ErrorHandlingDeserializer2将在包含原因和原始字节的标头中返回空值和DeserializationException。当您使用记录级别的MessageListener时，如果ConsumerRecord包含键或值的DeserializationException标头，则会以失败的{{1 }}。记录不会传递给侦听器。

相关问题
最新问题