Question

我们的Kafka连接日志经常出现此错误。（将我的主题名称替换为，等等）

[2020-02-20 06:22:22,169] ERROR WorkerSinkTask{id=s3connector-quoteandbind-0} Commit of offsets threw an unexpected exception for sequence number 288: 
{<topicname1>-0=OffsetAndMetadata{offset=0, leaderEpoch=null, metadata=''}, <topicname2>-0=OffsetAndMetadata{offset=6096, leaderEpoch=null, metadata=''}, <topicname3>-0=OffsetAndMetadata{offset=8023, leaderEpoch=null, metadata=''}, <topicname4>-0=OffsetAndMetadata{offset=6880, leaderEpoch=null, metadata=''}, 
<topicname5>-0=OffsetAndMetadata{offset=15745, leaderEpoch=null, metadata=''}, <topicname6>-0=OffsetAndMetadata{offset=6300, leaderEpoch=null, metadata=''}, <topicname7>-0=OffsetAndMetadata{offset=0, leaderEpoch=null, metadata=''}, <topicname8>-0=OffsetAndMetadata{offset=0, leaderEpoch=null, metadata=''}, <topicname9>-0=OffsetAndMetadata{offset=3875, leaderEpoch=null, metadata=''}, <topicname10>-0=OffsetAndMetadata{offset=5212, leaderEpoch=null, metadata=''}, <topicname11>-0=OffsetAndMetadata{offset=0, leaderEpoch=null, metadata=''}, <topicname12>-0=OffsetAndMetadata{offset=25761, leaderEpoch=null, metadata=''}, 
<topicname13>-0=OffsetAndMetadata{offset=768, leaderEpoch=null, metadata=''}, <topicname14>-0=OffsetAndMetadata{offset=30495, leaderEpoch=null, metadata=''}, <topicname15>-0=OffsetAndMetadata{offset=48136, leaderEpoch=null, metadata=''}, <topicname16>-0=OffsetAndMetadata{offset=31, leaderEpoch=null, metadata=''}, 
<topicname17>-0=OffsetAndMetadata{offset=6422, leaderEpoch=null, metadata=''}, <topicname18>-0=OffsetAndMetadata{offset=0, leaderEpoch=null, metadata=''}, <topicname19>-0=OffsetAndMetadata{offset=8471, leaderEpoch=null, metadata=''}, <topicname20>-0=OffsetAndMetadata{offset=22256, leaderEpoch=null, metadata=''}, <topicname21>-0=OffsetAndMetadata{offset=5520, leaderEpoch=null, metadata=''}, <topicname22>-0=OffsetAndMetadata{offset=29302, leaderEpoch=null, metadata=''}, <topicname23>-0=OffsetAndMetadata{offset=6880, leaderEpoch=null, metadata=''}, <topicname24>-0=OffsetAndMetadata{offset=21220, leaderEpoch=null, metadata=''}, 
<topicname25>-0=OffsetAndMetadata{offset=20881, leaderEpoch=null, metadata=''}, <topicname26>-0=OffsetAndMetadata{offset=144, leaderEpoch=null, metadata=''}, <topicname27>-0=OffsetAndMetadata{offset=3877, leaderEpoch=null, metadata=''}, <topicname28>-0=OffsetAndMetadata{offset=5324, leaderEpoch=null, metadata=''}, <topicname29>-0=OffsetAndMetadata{offset=5047, leaderEpoch=null, metadata=''}, <topicname30>-0=OffsetAndMetadata{offset=21637, leaderEpoch=null, metadata=''}, 
<topicname31>-0=OffsetAndMetadata{offset=2860, leaderEpoch=null, metadata=''}} (org.apache.kafka.connect.runtime.WorkerSinkTask)
************
************
timestamp: 2020-02-20T06:22:22.169Z
org.apache.kafka.clients.consumer.RetriableCommitFailedException: Offset commit failed with a retriable exception. You should retry committing the latest consumed offsets.
************

有人在此之后知道Kafka-Connect的行为吗？开发人员需要自动重试还是手动进行操作？

这是否意味着我们丢失了一些数据？

谢谢。

Answer 1

可重试异常是一个临时异常，如果重试可能会成功。

https://kafka.apache.org/20/javadoc/org/apache/kafka/common/errors/RetriableException.html

我不确定Connect是否会自行重启这些过程，但是S3连接器尤其具有一次语义，因此它应该继续重试相同的操作

“重试”的另一种方法是通过REST API重新启动连接器进程或连接器任务

Kafka Connect RetriableCommitFailedException-采取什么措施

1 个答案: