kafka spout不会发出数据

时间:2015-05-25 11:31:29

标签: java apache-kafka apache-storm kafka-consumer-api

我正在尝试将Kafka与风暴整合。我正在使用Kafka Spout从Kafka主题中检索数据并将其提供给风暴螺栓以进行进一步处理。我能够成功提交拓扑但是喷口没有发出任何data.It也不会抛出任何错误。我对Kafka和Storm很新。所以,我无法理解这个问题背后的原因。请建议修改。谢谢你提前!!

Screen Shot of Storm UI after submitting the topology

我的拓扑:

public class TopologyMain {

 private static final String SENTENCE_SPOUT_ID = "kafka-sentence-spout";


public static void main(String[] args) throws InterruptedException, AlreadyAliveException, InvalidTopologyException {
    int numSpoutExecutors = 1;


    TopologyBuilder builder = new TopologyBuilder();

    builder.setSpout(SENTENCE_SPOUT_ID, buildKafkaSentenceSpout(), numSpoutExecutors);
    builder.setBolt("word-normalizer", new WordNormalizer())
        .shuffleGrouping(SENTENCE_SPOUT_ID);
    builder.setBolt("word-counter", new WordCounter(),2)
        .shuffleGrouping("word-normalizer");

    //Configuration
    Config conf = new Config();
    conf.setDebug(false);
    //Topology run
    conf.put(Config.TOPOLOGY_MAX_SPOUT_PENDING, 1);
    conf.put(Config.NIMBUS_HOST, "192.168.1.229");
    conf.put(Config.NIMBUS_THRIFT_PORT, 6627);
    System.setProperty("storm.jar", "/home/ubuntu/st/stIn/target/storm-wc.jar");
    StormSubmitter.submitTopology("Count-Word-Topology", conf,builder.createTopology());

}



 private static KafkaSpout buildKafkaSentenceSpout() {
      BrokerHosts hosts = new ZkHosts("localhost:2181");
      SpoutConfig spoutConfig = new SpoutConfig(hosts, "test", "/acking-kafka-sentence-spout", "acking-sentence-spout");
      spoutConfig.forceFromStart = true;
      spoutConfig.startOffsetTime = kafka.api.OffsetRequest.EarliestTime();
      return new KafkaSpout(spoutConfig);
    }
 }

1 个答案:

答案 0 :(得分:0)

我明确地将我项目的Maven依赖项中的所有jar都复制到了Storm库中,并且工作正常。我还将风暴jar(用于提交拓扑的jar)复制到storm / lib。 / p>