Question

我们有一个具有多个kafka spout任务的拓扑。每个spout任务都应该从一组Kafka主题中读取一部分消息。必须使用诸如AAA.BBB。*之类的通配符订阅主题。预期的行为是所有spout任务将共同消耗与通配符匹配的所有主题中的所有消息。每条消息仅路由到一个spout任务（忽略故障情形）。目前是否支持？

Answer 1

也许您可以使用DynamicBrokersReader类。

Map conf = new HashMap();
...
conf.put("kafka.topic.wildcard.match", true);

wildCardBrokerReader = new DynamicBrokersReader(conf, connectionString, masterPath, "AAA.BBB.*");
List<GlobalPartitionInformation> partitions = wildCardBrokerReader.getBrokerInfo();
...
for (GlobalPartitionInformation eachTopic: partitions) {
    StaticHosts hosts = new StaticHosts(eachTopic);
    SpoutConfig spoutConfig = new SpoutConfig(hosts, eachTopic.topic, zkRoot, id);
    KafkaSpout spout = new KafkaSpout(spoutConfig);
}
... // Wrap those created spout instances into a container

Apache Storm：支持Kafka spout中的主题通配符

1 个答案: