Question

几天后，我尝试了将主题动态传递给Kafka监听器的方法，而不是通过Java DSL中的密钥使用它们。周围的人都是这样做的，或者可以说明实现这个目标的最佳方法是什么？

Answer 1

你不能动态地将主题传递给Kafka听众＆＃34 ;;您必须以编程方式创建一个侦听器容器。

Answer 2

我发现最简单的解决方案是使用SpEL：

p {
  color: #333;
  font-size: 12px !important;
  line-height: 18px !important;
  margin: 0 !important;
}

.content p{
  color: unset !important;
  font-size: initial !important;
  line-height: 25px !important;
  margin: initial !important;
}

Answer 3

您可以动态更改主题!!!!

@Component
public class StoppingErrorHandler implements ErrorHandler {

    @Autowired
    private KafkaListenerEndpointRegistry kafkaListenerEndpointRegistry;

    @Override
    public void handle(Exception thrownException, ConsumerRecord<?, ?> record) {
        ConcurrentMessageListenerContainer listenerContainer = (ConcurrentMessageListenerContainer)kafkaListenerEndpointRegistry.getListenerContainer("fence");
        ContainerProperties cp=listenerContainer.getContainerProperties();
        String[] topics =cp.getTopics();
        topics[0]="gaonb";
        listenerContainer.stop();
        listenerContainer.start();
    }
}

Answer 4

这是一个可行的解决方案：

// Start brokers without using the "@KafkaListener" annotation
Map<String, Object> consumerProps = consumerProps("my-srv1:9092", "my-group", "false");
DefaultKafkaConsumerFactory<String, String> cf = new DefaultKafkaConsumerFactory<>(consumerProps);
ContainerProperties containerProperties = new ContainerProperties("my-topic");
KafkaMessageListenerContainer container = new KafkaMessageListenerContainer<>(cf, containerProperties);
final BlockingQueue<ConsumerRecord<String, String>> records = new LinkedBlockingQueue<>();
container.setupMessageListener((MessageListener<String, String>) record -> {
    log.error("Message received: " + record);
    records.add(record);
});
container.start();

/**
 * Set up test properties for an {@code <Integer, String>} consumer.
 * @param brokersCommaSep the bootstrapServers property (comma separated servers).
 * @param group the group id.
 * @param autoCommit the auto commit.
 * @return the properties.
 */
public static Map<String, Object> consumerProps(String brokersCommaSep, String group, String autoCommit) {
    Map<String, Object> props = new HashMap<>();
    props.put(ConsumerConfig.BOOTSTRAP_SERVERS_CONFIG, brokersCommaSep);
    props.put(ConsumerConfig.GROUP_ID_CONFIG, group);
    props.put(ConsumerConfig.ENABLE_AUTO_COMMIT_CONFIG, autoCommit);
    props.put(ConsumerConfig.AUTO_COMMIT_INTERVAL_MS_CONFIG, "10");
    props.put(ConsumerConfig.SESSION_TIMEOUT_MS_CONFIG, 60000);
    props.put(ConsumerConfig.KEY_DESERIALIZER_CLASS_CONFIG, IntegerDeserializer.class);
    props.put(ConsumerConfig.VALUE_DESERIALIZER_CLASS_CONFIG, StringDeserializer.class);
    return props;
}

希望它可以提供帮助。

Answer 5

我为运行时注册，注销，启动，停止做了kafka侦听器。

public class KafkaListener {

     private final KafkaListenerContainerFactory kafkaListenerContainerFactory;
     private final Map<String, MessageListenerContainer> registeredTopicMap;

/** Kafka listener registration at runtime.**/
public void register(final Supplier<Set<String>> topicSupplier, final Supplier<MessageListener> messageListenerSupplier) {

    synchronized (lock) {
        final Set<String> registeredTopics = getRegisteredTopics();
        final Set<String> topics = topicSupplier.get();

        if (topics.isEmpty()) {
            return;
        }

        topics.stream()
                .filter(topic -> !registeredTopics.contains(topic))
                .forEach(topic -> doRegister(topic, messageListenerSupplier.get()));
    }
}


private void doRegister(final String topic, final MessageListener messageListener) {
    final MessageListenerContainer messageListenerContainer = kafkaListenerContainerFactory.createContainer(topic);
    messageListenerContainer.setupMessageListener(messageListener);
    messageListenerContainer.start();

    registeredTopicMap.put(topic, messageListenerContainer);
}

完整源代码：https://github.com/pkgonan/kafka-listener

首先，尝试一下。

docker-compose up -d

然后。呼叫api。

curl -XPOST /consumers/order/register .....

curl -XPOST /consumers/order/de-register .....

curl -XPOST /consumers/order/stop

curl -XPOST /consumers/order/start

如何动态地将主题传递给kafka监听器？

5 个答案: