使用storm-kafka-0.8-plus,只有少数喷口实际上可以提取数据

时间:2015-02-08 16:16:13

标签: java apache-kafka apache-storm

我一直在使用storm-kafka-0.8-plus和kafka 0.8.1.1&风暴0.9.0.1(zmq原因。)。我还使用了其他版本风暴的喷口。但问题是只有少数线程实际工作而其他线程仍处于空闲状态。我不知道为什么会这样。

public static void main(String[] args) throws Exception {
    brokerHosts = new ZkHosts(CONFIGURATION.getString("ZK_HOSTS") + "/"
            + CONFIGURATION.getString("ZK_CONNECT_PATH"));
    SpoutConfig spoutConfig = new SpoutConfig(brokerHosts,
            CONFIGURATION.getString("KAFKA_TOPIC"),
            CONFIGURATION.getString("ZOOKEEPER_STORAGE_PATH"),
            CONFIGURATION.getString("ZOOKEEPER_STORAGE_ID"));
    spoutConfig.startOffsetTime = kafka.api.OffsetRequest.LatestTime();
    KafkaSpout kafkaspout = new KafkaSpout(spoutConfig);

    TopologyBuilder builder = new TopologyBuilder();
    builder.setSpout("kafka-spout", kafkaspout,
            CONFIGURATION.getInt("NUM_KAFKA_SPOUT_THREADS"));

    Config config = new Config();
    StormSubmitter.submitTopology(CONFIGURATION.getString("TOPOLOGY_NAME"),
            config, builder.createTopology());
}

的pom.xml

<dependency>
  <groupId>storm</groupId>
  <artifactId>storm</artifactId>
  <version>0.9.0.1</version>
  <scope>provided</scope>
</dependency>
<dependency>
  <groupId>net.wurstmeister.storm</groupId>
  <artifactId>storm-kafka-0.8-plus</artifactId>
  <version>0.4.0</version>
</dependency>
<dependency>
  <groupId>org.apache.kafka</groupId>
  <artifactId>kafka_2.10</artifactId>
  <version>0.8.1</version>
</dependency> 

0 个答案:

没有答案