为什么我的kafka消费者心跳失败?

时间:2019-09-24 15:06:54

标签: apache-kafka heartbeat

我更新了一个C#使用者,它(似乎)随机进入了失败的心跳检查循环,一次平衡了几个小时才最终稳定下来。

我们有一个系统,该系统具有约20个不同的消费者,涉及大约12个主题,他们的生产稳定了几个月。我们尚未将任何使用者的session.timeout.ms或heartbeat.interval.ms更改为默认值。我最近更改的是其中一位消费者在做什么的胆量。它曾经用来更新一些Elasticsearch对象,但是现在它查询一个MSSQL数据库,然后更新elasticsearch。因此,我想知道消费者是否花了比以前更长的时间来运行并引起问题。但是,如果心跳在自己的线程上,那应该没关系,对吧?每个使用者有一个实例,只有一个分区。还有哪些其他配置信息可以帮助您了解?

我不知道显示什么代码来允许复制该代码。如果有人想看东西,请告诉我。

我们每8-9分钟收到一次此错误。上次发生的原因是它连续进行了4个小时多一点。这是两轮错误:

[2019-09-24 06:05:00,832] INFO [GroupCoordinator 0]: Member rdkafka-5e17a7b1-fb41-4249-887a-1e2a0599f2d7 in group TV.Next.LoadScanElasticSearchConsumer has failed, removing it from the group (kafka.coordinator.group.GroupCoordinator)
[2019-09-24 06:05:00,832] INFO [GroupCoordinator 0]: Preparing to rebalance group TV.Next.LoadScanElasticSearchConsumer in state PreparingRebalance with old generation 773 (__consumer_offsets-41) (reason: removing member rdkafka-5e17a7b1-fb41-4249-887a-1e2a0599f2d7 on heartbeat expiration) (kafka.coordinator.group.GroupCoordinator)
[2019-09-24 06:05:00,832] INFO [GroupCoordinator 0]: Group TV.Next.LoadScanElasticSearchConsumer with generation 774 is now empty (__consumer_offsets-41) (kafka.coordinator.group.GroupCoordinator)
[2019-09-24 06:05:31,098] INFO [GroupCoordinator 0]: Preparing to rebalance group TV.Next.LoadScanElasticSearchConsumer in state PreparingRebalance with old generation 774 (__consumer_offsets-41) (reason: Adding new member rdkafka-ee8a935c-384d-4e3b-93ad-85d208790d22 with group instanceid None) (kafka.coordinator.group.GroupCoordinator)
[2019-09-24 06:05:31,113] INFO [GroupCoordinator 0]: Stabilized group TV.Next.LoadScanElasticSearchConsumer generation 775 (__consumer_offsets-41) (kafka.coordinator.group.GroupCoordinator)
[2019-09-24 06:05:31,129] INFO [GroupCoordinator 0]: Assignment received from leader for group TV.Next.LoadScanElasticSearchConsumer for generation 775 (kafka.coordinator.group.GroupCoordinator)
[2019-09-24 06:08:35,650] INFO [GroupMetadataManager brokerId=0] Removed 0 expired offsets in 0 milliseconds. (kafka.coordinator.group.GroupMetadataManager)
[2019-09-24 06:13:48,132] INFO [GroupCoordinator 0]: Member rdkafka-ee8a935c-384d-4e3b-93ad-85d208790d22 in group TV.Next.LoadScanElasticSearchConsumer has failed, removing it from the group (kafka.coordinator.group.GroupCoordinator)
[2019-09-24 06:13:48,132] INFO [GroupCoordinator 0]: Preparing to rebalance group TV.Next.LoadScanElasticSearchConsumer in state PreparingRebalance with old generation 775 (__consumer_offsets-41) (reason: removing member rdkafka-ee8a935c-384d-4e3b-93ad-85d208790d22 on heartbeat expiration) (kafka.coordinator.group.GroupCoordinator)
[2019-09-24 06:13:48,132] INFO [GroupCoordinator 0]: Group TV.Next.LoadScanElasticSearchConsumer with generation 776 is now empty (__consumer_offsets-41) (kafka.coordinator.group.GroupCoordinator)
[2019-09-24 06:14:19,647] INFO [GroupCoordinator 0]: Preparing to rebalance group TV.Next.LoadScanElasticSearchConsumer in state PreparingRebalance with old generation 776 (__consumer_offsets-41) (reason: Adding new member rdkafka-5ddb9247-68f0-415d-a702-beca62c2aa2b with group instanceid None) (kafka.coordinator.group.GroupCoordinator)
[2019-09-24 06:14:19,663] INFO [GroupCoordinator 0]: Stabilized group TV.Next.LoadScanElasticSearchConsumer generation 777 (__consumer_offsets-41) (kafka.coordinator.group.GroupCoordinator)
[2019-09-24 06:14:19,679] INFO [GroupCoordinator 0]: Assignment received from leader for group TV.Next.LoadScanElasticSearchConsumer for generation 777 (kafka.coordinator.group.GroupCoordinator)
[2019-09-24 06:18:35,646] INFO [GroupMetadataManager brokerId=0] Removed 0 expired offsets in 0 milliseconds. (kafka.coordinator.group.GroupMetadataManager)

0 个答案:

没有答案