Question

public class WordCountTopology {
   private static final String FILTER_BOLT_ID = "filter-bolt";
   private static final String COUNT_BOLT_ID = "count-bolt";
   private static final String TOPOLOGY_NAME = "word-count-topology";

static final PropertiesConfiguration CONFIGURATION = new PropertiesConfiguration();

static {
try {
  CONFIGURATION.load(ClassLoader.getSystemResourceAsStream(
      "config/topo.properties"));
  } catch (ConfigurationException e) {
  throw new RuntimeException("Failed to load topo properties");
  }
}


private static KafkaSpout createKafkaSpout() {
  List<String> hostList = new ArrayList<String>();
  String kafkaPortStr = CONFIGURATION.getString("KAFKA_PORT");
  String[] kafkaHosts = CONFIGURATION.getString("KAFKA_HOSTS").split(",");
  for (String kafkaHost : kafkaHosts) {
    hostList.add(kafkaHost + ":" + kafkaPortStr);
  }
  BrokerHosts hosts = StaticHosts.fromHostString(
    hostList, CONFIGURATION.getInt("NUM_KAFKA_PARTITIONS"));

  SpoutConfig spoutConfig = new SpoutConfig(
    hosts,
    CONFIGURATION.getString("KAFKA_TOPIC"),
    CONFIGURATION.getString("ZOOKEEPER_STORAGE_PATH"),
    CONFIGURATION.getString("ZOOKEEPER_STORAGE_ID"));
    spoutConfig.forceStartOffsetTime(-1);

  return new KafkaSpout(spoutConfig);
}


topo.properties (not full file):

Topology1
KAFKA_TOPIC=varnish
ZOOKEEPER_STORAGE_PATH=/kafkastorm
ZOOKEEPER_STORAGE_ID=discovery

Topology2
KAFKA_TOPIC=varnish
ZOOKEEPER_STORAGE_PATH=/kafkastorm8
ZOOKEEPER_STORAGE_ID=discovery8

我正在尝试运行两个使用相同kafka主题的数据的tpoplogies，拓扑1可以正常使用与上面类似的代码但拓扑2给出了错误。

我收到kafka spout的以下错误

kafka.common.OffsetOutOfRangeException at sun.reflect.NativeConstructorAccessorImpl.newInstance0（Native Method）at sun.reflect.NativeConstructorAccessorImpl.newInstance（NativeConstructorAccessorImpl

Answer 1

你可以试着改变一下，看看它是否有效 spoutConfig.forceStartOffsetTime(-1);
至
spoutConfig.forceStartOffsetTime(-2);
这会强制鲸鱼喷水回流到之前的偏移，有关详细信息，您可以read this page

Answer 2

解决方案是我必须去动物园管理员并使用zookeeper cli我删除了拓扑2的zk存储根路径/ kafkastorm8并重新提交拓扑并且它工作正常。

在相同的kafka主题（使用kafka-spout）读取的不同群集上运行的两个独立的风暴拓扑给出了Error

2 个答案: