两个经纪人之一关闭后,经纪人可能不可用

时间:2019-12-06 19:37:44

标签: windows spring-boot apache-kafka replication-factor

我在Windows上的Spring MVC的spring-boot命令行项目中使用REST API。我有两个经纪人,一个主题,一个生产者和两个消费者,像这样:

Topic:velib-stations    PartitionCount:10       ReplicationFactor:2     Configs:
    Topic: velib-stations   Partition: 0    Leader: 1       Replicas: 1,0   Isr: 0,1
    Topic: velib-stations   Partition: 1    Leader: 0       Replicas: 0,1   Isr: 0,1
    Topic: velib-stations   Partition: 2    Leader: 1       Replicas: 1,0   Isr: 0,1
    Topic: velib-stations   Partition: 3    Leader: 0       Replicas: 0,1   Isr: 0,1
    Topic: velib-stations   Partition: 4    Leader: 1       Replicas: 1,0   Isr: 0,1
    Topic: velib-stations   Partition: 5    Leader: 0       Replicas: 0,1   Isr: 0,1
    Topic: velib-stations   Partition: 6    Leader: 1       Replicas: 1,0   Isr: 0,1
    Topic: velib-stations   Partition: 7    Leader: 0       Replicas: 0,1   Isr: 0,1
    Topic: velib-stations   Partition: 8    Leader: 1       Replicas: 1,0   Isr: 0,1
    Topic: velib-stations   Partition: 9    Leader: 0       Replicas: 0,1   Isr: 0,1

当我关闭一位经纪人时,它应该重新平衡到剩下的一位。但这种情况并非如此。我的Java控制台正在轰炸我:

[Consumer clientId=consumer-1, groupId=velib-group] Connection to node 0 (localhost/127.0.0.1:9092) could not be established. Broker may not be available.

当只有一个平衡时,平衡会像预期的那样艰难:

Topic:velib-stations    PartitionCount:10       ReplicationFactor:2     Configs:
    Topic: velib-stations   Partition: 0    Leader: 0       Replicas: 1,0   Isr: 0
    Topic: velib-stations   Partition: 1    Leader: 0       Replicas: 0,1   Isr: 0
    Topic: velib-stations   Partition: 2    Leader: 0       Replicas: 1,0   Isr: 0
    Topic: velib-stations   Partition: 3    Leader: 0       Replicas: 0,1   Isr: 0
    Topic: velib-stations   Partition: 4    Leader: 0       Replicas: 1,0   Isr: 0
    Topic: velib-stations   Partition: 5    Leader: 0       Replicas: 0,1   Isr: 0
    Topic: velib-stations   Partition: 6    Leader: 0       Replicas: 1,0   Isr: 0
    Topic: velib-stations   Partition: 7    Leader: 0       Replicas: 0,1   Isr: 0
    Topic: velib-stations   Partition: 8    Leader: 0       Replicas: 1,0   Isr: 0
    Topic: velib-stations   Partition: 9    Leader: 0       Replicas: 0,1   Isr: 0

我已经设法通过在两台服务器上设置属性来解决NOT_LEADER_FOR_PARTITION错误:

leader.imbalance.check.interval.seconds=1 
unclean.leader.election.enable=true

我将这个属性设置为生产者和消费者

properties.setProperty(ConsumerConfig.METADATA_MAX_AGE_CONFIG, "1");

最后这是两台服务器(在9092和9093上)的属性中的网络配置

listeners=PLAINTEXT://localhost:9092
advertised.listeners=PLAINTEXT://localhost:9092

如何解决代理不可用以及能够将所有消息重定向到其余代理的问题?也许默认行为无法实现?谢谢

编辑:有人要求我分享consumer_offset,但是当两个服务器都向上或向下时,它总是显示以下内容:

__consumer_offsets
velib-stations

编辑2 :我尝试了相同的程序,但是在Linux上使用python kafka库,存在相同的问题。所以这是一个卡夫卡问题:(

0 个答案:

没有答案