Question

我正在尝试将项目从apache-storm迁移到twitter-heron。经过大量的努力，我能够摆脱大多数错误，例如使用className: "org.apache.storm.kafka.ZkHosts"而不是className: "storm.kafka.ZkHosts"。但是，我一直在提交拓扑。我使用Flux将拓扑提交给风暴。

我在ZkState中创建CuratorFramework对象时遇到NullPointerException。在进一步挖掘时，我发现an issue in github，如果未设置有关zookeeper的配置，则会导致此问题。

进一步调试我发现问题是因为我缺少ZkState.java:46中所需的以下配置。

storm.zookeeper.session.timeout
storm.zookeeper.connection.timeout
storm.zookeeper.retry.times
storm.zookeeper.retry.interval

虽然我已设法识别问题，但是，我不知道在我的配置中将其添加到何处。有人可以帮我在哪里添加上述配置。谢谢。

My Flux Config

name: "My_Topology"
components:
  - id: "zkHosts"
    className: "org.apache.storm.kafka.ZkHosts"
    constructorArgs:
      - "localhost:2181"

  - id: "SpoutConfig"
    className: "org.apache.storm.kafka.SpoutConfig"
    constructorArgs:
      - ref: "zkHosts" # brokerHosts
      - "my-topic"  # topic
      - "/my-zkRoot" # zkRoot
      - "my-id" # spoutId
    properties: 
      - name: "zkServers"
        value: ["localhost"]
      - name: "zkPort"
        value: 2181
      - name: "zkRoot"
        value: "/my-zkRoot"
      - name: "retryInitialDelayMs"
        value: 2000
      - name: "retryDelayMultiplier"
        value: 2

config:
  topology.workers: 5
  topology.testing.always.try.serialize: true

spouts:
  - id: "kafka-spout"
    className: "org.apache.storm.kafka.KafkaSpout"
    parallelism: 1
    constructorArgs:
      - ref: "SpoutConfig"
bolts:
  - id: "my-bolt"
    className: "com.example.sample.MyBolt"
    parallelism: 1

streams:
  - name: "kafka_spout --> my_bolt"
    from: "kafka-spout"
    to: "my-bolt"
    grouping:
      type: SHUFFLE

Answer 1

您可以将这些添加到您的Flux yaml文件的配置部分

config:
  topology.workers: 5
  topology.testing.always.try.serialize: true
  storm.zookeeper.session.timeout: 30000
  storm.zookeeper.connection.timeout: 30000
  storm.zookeeper.retry.times: 5
  storm.zookeeper.retry.interval: 2000

Answer 2

我们已经将Storm Flux本地集成到Heron中以简化它。使用Heron ECO - 您可以编写两种不同API的拓扑

Native Heron API
Storm API

有关其他信息，请参阅此处的ECO文档

https://apache.github.io/incubator-heron/docs/developers/java/eco-api/

如何使用Twitter Heron与Storm Flux

2 个答案: