我正在开发一个Springboot应用程序,它在Kafka主题(例如700个主题,每个主题有10个分区,即7000个分区)中重新平衡。但我想启动多个dockerized应用程序实例,其中该应用程序将包含所有700个主题名称,但它应仅选择前25个分区,然后取消选择其他分区)
@KafkaListener( topics = "#{kafkaProperties.getTopics()}" )
kafkaProperties.getTopics()返回所有700个主题名称
答案 0 :(得分:0)
正如我在最初的评论中要求澄清的那样,对于这样的分区分配方案,您将需要实现自定义useEffect(() => {
const onAnswerCall = () => {
console.log('=== onAnswerCall ::: ===', calls);
};
RNCallKeep.addEventListener('answerCall', onAnswerCall);
return () => {
RNCallKeep.removeEventListener('answerCall', onAnswerCall);
};
}, [calls]);
。
每个实例应选择数量有限的线程,每个线程具有一个主题/分区
实例不“挑选”主题/分区;选择一个实例,并确定哪个实例获得哪个主题/分区。
请参阅其javadocs。
ConsumerPartitionAssignor
要将更多线程(消费者)添加到实例,请增加容器并发性。
对于大量实例,我建议使用/**
* This interface is used to define custom partition assignment for use in
* {@link org.apache.kafka.clients.consumer.KafkaConsumer}. Members of the consumer group subscribe
* to the topics they are interested in and forward their subscriptions to a Kafka broker serving
* as the group coordinator. The coordinator selects one member to perform the group assignment and
* propagates the subscriptions of all members to it. Then {@link #assign(Cluster, GroupSubscription)} is called
* to perform the assignment and the results are forwarded back to each respective members
*
* In some cases, it is useful to forward additional metadata to the assignor in order to make
* assignment decisions. For this, you can override {@link #subscriptionUserData(Set)} and provide custom
* userData in the returned Subscription. For example, to have a rack-aware assignor, an implementation
* can use this user data to forward the rackId belonging to each member.
*/
public interface ConsumerPartitionAssignor {
重新平衡协议。