Question

我正在运行Kafka Stream应用程序，最近我开始遇到以下异常，并且Kafka Stream进程进入挂起的关闭状态。这显示了kafka生产者内部API代码中的异常。可能是因为kafka经纪人负担沉重吗？

2019-08-12 10:54:30 - [ERROR] [kafka-producer-network-thread | c8-max-view-live-1-StreamThread-1-producer] [org.apache.kafka.clients.producer.internals.Sender.run:235] : [Producer clientId=c8-max-view-live-1-StreamThread-1-producer] Uncaught error in kafka producer I/O thread:
java.util.ConcurrentModificationException: null
    at java.util.HashMap$HashIterator.nextNode(HashMap.java:1429)
    at java.util.HashMap$EntryIterator.next(HashMap.java:1463)
    at java.util.HashMap$EntryIterator.next(HashMap.java:1461)
    at org.apache.kafka.clients.producer.internals.Sender.getExpiredInflightBatches(Sender.java:177)
    at org.apache.kafka.clients.producer.internals.Sender.sendProducerData(Sender.java:353)
    at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:308)
    at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:233)
    at java.lang.Thread.run(Thread.java:745)
2019-08-12 10:54:30 - [ERROR] [kafka-producer-network-thread | c8-max-view-live-1-StreamThread-1-producer] [org.apache.kafka.clients.producer.internals.Sender.run:235] : [Producer clientId=c8-max-view-live-1-StreamThread-1-producer] Uncaught error in kafka producer I/O thread:
java.util.ConcurrentModificationException: null
    at java.util.HashMap$HashIterator.nextNode(HashMap.java:1429)
    at java.util.HashMap$EntryIterator.next(HashMap.java:1463)
    at java.util.HashMap$EntryIterator.next(HashMap.java:1461)
    at org.apache.kafka.clients.producer.internals.Sender.getExpiredInflightBatches(Sender.java:177)
    at org.apache.kafka.clients.producer.internals.Sender.sendProducerData(Sender.java:353)
    at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:308)
    at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:233)
    at java.lang.Thread.run(Thread.java:745)

此卡夫卡流处理过程挂起后：

2019-08-12 10:54:31 - [INFO] [c8-max-view-live-1-StreamThread-1] [org.apache.kafka.streams.KafkaStreams.setState:257] : stream-client [c8-max-view-live-1] State transition from ERROR to PENDING_SHUTDOWN
2019-08-12 10:54:31 - [INFO] [kafka-streams-close-thread] [org.apache.kafka.streams.processor.internals.StreamThread.shutdown:1164] : stream-thread [c8-max-view-live-1-StreamThread-1] Informed to shut down

Answer 1

围绕此问题的一个错误已确定并已修复，该错误在迭代时更改基础Collection。请检查here。

Kafka Stream-Kafka生产者I / O线程中发生未捕获的错误：java.util.ConcurrentModificationException：null

1 个答案: