我在docker上使用kafka（注意：带有wurstmeister图像）。 我有三个消费者组（每个组包含3个消费者）。
两组正在运行，没有问题，最后一组正在运行不良（许多在kafka日志上重新平衡）：消费者代码与group1相同（复制/粘贴KafkaConsumer代码）
我在group3的3个使用者中添加了日志，但是我无法确定问题所在：
2020-08-10 10:00:35 DEBUG <BrokerConnection node_id=coordinator-1001 host=kafka:9092 <connected> [IPv4 ('172.18.0.6', 9092)]> Request 191: HeartbeatRequest_v1(group='group3', generation_id=2, member_id='kafka-python-2.0.1-8828091f-190e-49bb-88e3-1e702b221547')
2020-08-10 10:00:35 DEBUG Received correlation id: 191
2020-08-10 10:00:35 DEBUG Processing response HeartbeatResponse_v1
2020-08-10 10:00:35 DEBUG <BrokerConnection node_id=coordinator-1001 host=kafka:9092 <connected> [IPv4 ('172.18.0.6', 9092)]> Response 191 (101.25017166137695 ms): HeartbeatResponse_v1(throttle_time_ms=0, error_code=0)
2020-08-10 10:00:35 DEBUG Received successful heartbeat response for group group3
2020-08-10 10:00:37 DEBUG Heartbeat: group3[2] kafka-python-2.0.1-8828091f-190e-49bb-88e3-1e702b221547
2020-08-10 10:00:37 DEBUG Sending request HeartbeatRequest_v1(group='group3', generation_id=2, member_id='kafka-python-2.0.1-8828091f-190e-49bb-88e3-1e702b221547')
2020-08-10 10:00:37 DEBUG <BrokerConnection node_id=coordinator-1001 host=kafka:9092 <connected> [IPv4 ('172.18.0.6', 9092)]> Request 192: HeartbeatRequest_v1(group='group3', generation_id=2, member_id='kafka-python-2.0.1-8828091f-190e-49bb-88e3-1e702b221547')
2020-08-10 10:00:37 DEBUG Received correlation id: 192
2020-08-10 10:00:37 DEBUG Processing response HeartbeatResponse_v1
2020-08-10 10:00:37 DEBUG <BrokerConnection node_id=coordinator-1001 host=kafka:9092 <connected> [IPv4 ('172.18.0.6', 9092)]> Response 192 (101.10831260681152 ms): HeartbeatResponse_v1(throttle_time_ms=0, error_code=0)
2020-08-10 10:00:37 DEBUG Received successful heartbeat response for group group3
2020-08-10 10:00:39 DEBUG Heartbeat: group3[2] kafka-python-2.0.1-8828091f-190e-49bb-88e3-1e702b221547
2020-08-10 10:00:39 DEBUG Sending request HeartbeatRequest_v1(group='group3', generation_id=2, member_id='kafka-python-2.0.1-8828091f-190e-49bb-88e3-1e702b221547')
2020-08-10 10:00:39 DEBUG <BrokerConnection node_id=coordinator-1001 host=kafka:9092 <connected> [IPv4 ('172.18.0.6', 9092)]> Request 193: HeartbeatRequest_v1(group='group3', generation_id=2, member_id='kafka-python-2.0.1-8828091f-190e-49bb-88e3-1e702b221547')
2020-08-10 10:00:40 DEBUG Received correlation id: 193
2020-08-10 10:00:40 DEBUG Processing response HeartbeatResponse_v1
2020-08-10 10:00:40 DEBUG <BrokerConnection node_id=coordinator-1001 host=kafka:9092 <connected> [IPv4 ('172.18.0.6', 9092)]> Response 193 (101.21679306030273 ms): HeartbeatResponse_v1(throttle_time_ms=0, error_code=0)
2020-08-10 10:00:40 DEBUG Received successful heartbeat response for group group3
2020-08-10 10:00:41 DEBUG Heartbeat: group3[2] kafka-python-2.0.1-8828091f-190e-49bb-88e3-1e702b221547
2020-08-10 10:00:42 DEBUG Sending request HeartbeatRequest_v1(group='group3', generation_id=2, member_id='kafka-python-2.0.1-8828091f-190e-49bb-88e3-1e702b221547')
2020-08-10 10:00:42 DEBUG <BrokerConnection node_id=coordinator-1001 host=kafka:9092 <connected> [IPv4 ('172.18.0.6', 9092)]> Request 194: HeartbeatRequest_v1(group='group3', generation_id=2, member_id='kafka-python-2.0.1-8828091f-190e-49bb-88e3-1e702b221547')
2020-08-10 10:00:42 DEBUG Received correlation id: 194
2020-08-10 10:00:42 DEBUG Processing response HeartbeatResponse_v1
2020-08-10 10:00:42 DEBUG <BrokerConnection node_id=coordinator-1001 host=kafka:9092 <connected> [IPv4 ('172.18.0.6', 9092)]> Response 194 (103.01470756530762 ms): HeartbeatResponse_v1(throttle_time_ms=0, error_code=0)
2020-08-10 10:00:42 DEBUG Received successful heartbeat response for group group3
2020-08-10 10:00:43 DEBUG Heartbeat: group3[2] kafka-python-2.0.1-8828091f-190e-49bb-88e3-1e702b221547
2020-08-10 10:00:43 DEBUG Sending request HeartbeatRequest_v1(group='group3', generation_id=2, member_id='kafka-python-2.0.1-8828091f-190e-49bb-88e3-1e702b221547')
2020-08-10 10:00:43 DEBUG <BrokerConnection node_id=coordinator-1001 host=kafka:9092 <connected> [IPv4 ('172.18.0.6', 9092)]> Request 195: HeartbeatRequest_v1(group='group3', generation_id=2, member_id='kafka-python-2.0.1-8828091f-190e-49bb-88e3-1e702b221547')
2020-08-10 10:00:44 DEBUG Received correlation id: 195
2020-08-10 10:00:44 DEBUG Processing response HeartbeatResponse_v1
2020-08-10 10:00:44 DEBUG <BrokerConnection node_id=coordinator-1001 host=kafka:9092 <connected> [IPv4 ('172.18.0.6', 9092)]> Response 195 (101.14407539367676 ms): HeartbeatResponse_v1(throttle_time_ms=0, error_code=0)
2020-08-10 10:00:44 DEBUG Received successful heartbeat response for group group3
2020-08-10 10:00:45 DEBUG Heartbeat: group3[2] kafka-python-2.0.1-8828091f-190e-49bb-88e3-1e702b221547
2020-08-10 10:00:45 DEBUG Sending request HeartbeatRequest_v1(group='group3', generation_id=2, member_id='kafka-python-2.0.1-8828091f-190e-49bb-88e3-1e702b221547')
2020-08-10 10:00:45 DEBUG <BrokerConnection node_id=coordinator-1001 host=kafka:9092 <connected> [IPv4 ('172.18.0.6', 9092)]> Request 196: HeartbeatRequest_v1(group='group3', generation_id=2, member_id='kafka-python-2.0.1-8828091f-190e-49bb-88e3-1e702b221547')
2020-08-10 10:00:46 DEBUG Received correlation id: 196
2020-08-10 10:00:46 DEBUG Processing response HeartbeatResponse_v1
2020-08-10 10:00:46 DEBUG <BrokerConnection node_id=coordinator-1001 host=kafka:9092 <connected> [IPv4 ('172.18.0.6', 9092)]> Response 196 (100.91781616210938 ms): HeartbeatResponse_v1(throttle_time_ms=0, error_code=0)
2020-08-10 10:00:46 DEBUG Received successful heartbeat response for group group3
2020-08-10 10:00:47 WARNING Heartbeat poll expired, leaving group
2020-08-10 10:00:47 INFO Leaving consumer group (group3).
2020-08-10 10:00:47 DEBUG Sending request LeaveGroupRequest_v1(group='group3', member_id='kafka-python-2.0.1-8828091f-190e-49bb-88e3-1e702b221547')
2020-08-10 10:00:47 DEBUG <BrokerConnection node_id=coordinator-1001 host=kafka:9092 <connected> [IPv4 ('172.18.0.6', 9092)]> Request 197: LeaveGroupRequest_v1(group='group3', member_id='kafka-python-2.0.1-8828091f-190e-49bb-88e3-1e702b221547')
2020-08-10 10:00:47 DEBUG Received correlation id: 197
2020-08-10 10:00:47 DEBUG Processing response LeaveGroupResponse_v1
2020-08-10 10:00:47 DEBUG <BrokerConnection node_id=coordinator-1001 host=kafka:9092 <connected> [IPv4 ('172.18.0.6', 9092)]> Response 197 (11.553049087524414 ms): LeaveGroupResponse_v1(throttle_time_ms=0, error_code=0)
2020-08-10 10:00:47 DEBUG LeaveGroup request for group group3 returned successfully
2020-08-10 10:00:47 DEBUG Group state is not stable, disabling heartbeats
2020-08-10 10:00:47 DEBUG Heartbeat disabled. Waiting
有人有主意吗？谢谢！
编辑：使用者配置
self.consumer = KafkaConsumer(outTopic,
bootstrap_servers=brokers,
auto_offset_reset='earliest',
enable_auto_commit=True,
session_timeout_ms=14000,
heartbeat_interval_ms=2000,
group_id='group3',
key_deserializer=lambda x: loads(x.decode('utf-8')),
value_deserializer=lambda x: loads(x.decode('utf-8')))
EDIT2：警告显示在日志中（已过滤）=>心跳每2秒失败一次，并且每5分钟轮询一次心跳失败
2020-08-11 08:05:02 WARNING Heartbeat failed for group group3 because it is rebalancing
2020-08-11 08:05:04 WARNING Heartbeat failed for group group3 because it is rebalancing
2020-08-11 08:05:06 WARNING Heartbeat failed for group group3 because it is rebalancing
2020-08-11 08:05:08 WARNING Heartbeat failed for group group3 because it is rebalancing
2020-08-11 08:05:10 WARNING Heartbeat failed for group group3 because it is rebalancing
2020-08-11 08:05:12 WARNING Heartbeat failed for group group3 because it is rebalancing
2020-08-11 08:05:14 WARNING Heartbeat failed for group group3 because it is rebalancing
2020-08-11 08:10:15 WARNING Heartbeat poll expired, leaving group
2020-08-11 08:12:45 WARNING Heartbeat failed for group group3 because it is rebalancing
2020-08-11 08:12:47 WARNING Heartbeat failed for group group3 because it is rebalancing
2020-08-11 08:12:49 WARNING Heartbeat failed for group group3 because it is rebalancing
2020-08-11 08:12:51 WARNING Heartbeat failed for group group3 because it is rebalancing
2020-08-11 08:12:53 WARNING Heartbeat failed for group group3 because it is rebalancing
2020-08-11 08:12:55 WARNING Heartbeat failed for group group3 because it is rebalancing
编辑：kafka容器侧的重新平衡（我认为是由于轮询失败）
[2020-08-12 12:41:50,514] INFO [GroupCoordinator 1001]: Preparing to rebalance group group3 in state PreparingRebalance with old generation 115 (__consumer_offsets-38) (reason: removing member kafka-python-2.0.1-3d0acd18-76d4-4c6c-a1c6-a5e202a3d7c2 on LeaveGroup) (kafka.coordinator.group.GroupCoordinator)
[2020-08-12 12:41:50,547] INFO [GroupCoordinator 1001]: Member[group.instance.id None, member.id kafka-python-2.0.1-f5e4662b-d1d0-45c0-93db-c0f131a84b8b] in group group3 has left, removing it from the group (kafka.coordinator.group.GroupCoordinator)
[2020-08-12 12:41:50,564] INFO [GroupCoordinator 1001]: Group group3 with generation 116 is now empty (__consumer_offsets-38) (kafka.coordinator.group.GroupCoordinator)
[2020-08-12 12:42:40,681] INFO [GroupCoordinator 1001]: Dynamic Member with unknown member id joins group group3 in Empty state. Created a new member id kafka-python-2.0.1-a6120fb6-a528-46aa-9d6c-632a6343fd2c for this member and add to the group. (kafka.coordinator.group.GroupCoordinator)
[2020-08-12 12:42:40,683] INFO [GroupCoordinator 1001]: Preparing to rebalance group group3 in state PreparingRebalance with old generation 116 (__consumer_offsets-38) (reason: Adding new member kafka-python-2.0.1-a6120fb6-a528-46aa-9d6c-632a6343fd2c with group instance id None) (kafka.coordinator.group.GroupCoordinator)
[2020-08-12 12:42:40,686] INFO [GroupCoordinator 1001]: Stabilized group group3 generation 117 (__consumer_offsets-38) (kafka.coordinator.group.GroupCoordinator)
[2020-08-12 12:42:40,692] INFO [GroupCoordinator 1001]: Assignment received from leader for group group3 for generation 117 (kafka.coordinator.group.GroupCoordinator)
[2020-08-12 12:42:45,578] INFO [GroupCoordinator 1001]: Dynamic Member with unknown member id joins group group3 in Stable state. Created a new member id kafka-python-2.0.1-dc5e96eb-f612-4e14-9a06-62cc39ea6485 for this member and add to the group. (kafka.coordinator.group.GroupCoordinator)
[2020-08-12 12:42:45,579] INFO [GroupCoordinator 1001]: Preparing to rebalance group group3 in state PreparingRebalance with old generation 117 (__consumer_offsets-38) (reason: Adding new member kafka-python-2.0.1-dc5e96eb-f612-4e14-9a06-62cc39ea6485 with group instance id None) (kafka.coordinator.group.GroupCoordinator)
[2020-08-12 12:47:40,547] INFO [GroupMetadataManager brokerId=1001] Removed 0 expired offsets in 3 milliseconds. (kafka.coordinator.group.GroupMetadataManager)
[2020-08-12 12:47:45,584] INFO [GroupCoordinator 1001]: Member kafka-python-2.0.1-dc5e96eb-f612-4e14-9a06-62cc39ea6485 in group group3 has failed, removing it from the group (kafka.coordinator.group.GroupCoordinator)
[2020-08-12 12:47:45,616] INFO [GroupCoordinator 1001]: Dynamic Member with unknown member id joins group group3 in PreparingRebalance state. Created a new member id kafka-python-2.0.1-e2bbe292-9f91-4777-926a-de377e7eb543 for this member and add to the group. (kafka.coordinator.group.GroupCoordinator)
[2020-08-12 12:50:00,039] INFO [GroupCoordinator 1001]: Stabilized group group3 generation 118 (__consumer_offsets-38) (kafka.coordinator.group.GroupCoordinator)
[2020-08-12 12:50:00,078] INFO [GroupCoordinator 1001]: Assignment received from leader for group group3 for generation 118 (kafka.coordinator.group.GroupCoordinator)
注意：长时间运行后，轮询失败并重新平衡