我正在尝试将Kafka与风暴整合。我正在使用Kafka Spout从Kafka主题中检索数据并将其提供给风暴螺栓以进行进一步处理。我能够成功提交拓扑但是喷口没有发出任何data.It也不会抛出任何错误。我对Kafka和Storm很新。所以,我无法理解这个问题背后的原因。请建议修改。谢谢你提前!!
我的拓扑:
public class TopologyMain {
private static final String SENTENCE_SPOUT_ID = "kafka-sentence-spout";
public static void main(String[] args) throws InterruptedException, AlreadyAliveException, InvalidTopologyException {
int numSpoutExecutors = 1;
TopologyBuilder builder = new TopologyBuilder();
builder.setSpout(SENTENCE_SPOUT_ID, buildKafkaSentenceSpout(), numSpoutExecutors);
builder.setBolt("word-normalizer", new WordNormalizer())
.shuffleGrouping(SENTENCE_SPOUT_ID);
builder.setBolt("word-counter", new WordCounter(),2)
.shuffleGrouping("word-normalizer");
//Configuration
Config conf = new Config();
conf.setDebug(false);
//Topology run
conf.put(Config.TOPOLOGY_MAX_SPOUT_PENDING, 1);
conf.put(Config.NIMBUS_HOST, "192.168.1.229");
conf.put(Config.NIMBUS_THRIFT_PORT, 6627);
System.setProperty("storm.jar", "/home/ubuntu/st/stIn/target/storm-wc.jar");
StormSubmitter.submitTopology("Count-Word-Topology", conf,builder.createTopology());
}
private static KafkaSpout buildKafkaSentenceSpout() {
BrokerHosts hosts = new ZkHosts("localhost:2181");
SpoutConfig spoutConfig = new SpoutConfig(hosts, "test", "/acking-kafka-sentence-spout", "acking-sentence-spout");
spoutConfig.forceFromStart = true;
spoutConfig.startOffsetTime = kafka.api.OffsetRequest.EarliestTime();
return new KafkaSpout(spoutConfig);
}
}
答案 0 :(得分:0)
我明确地将我项目的Maven依赖项中的所有jar都复制到了Storm库中,并且工作正常。我还将风暴jar(用于提交拓扑的jar)复制到storm / lib。 / p>