Question

我使用kafka-storm，zookeeper（3.4.6）在Apache Storm（0.9.6）中创建了拓扑（每个节点3个zookeeper，每个节点3个主管。操作3个拓扑）

我添加了2个storm＆amp; zookeeper节点并更改了topology.worker配置3到5。但在2个节点之后，风暴主管经常关闭工人。使用iostat命令检查，读写吞吐量低于1mb。在主管日志中，如下所示。

2016-10-19T15:07:38.904+0900 b.s.d.supervisor [INFO] Shutting down and clearing state for id ee13ada9-641e-463a-9be5-f3ed66fdb8f3. Current supervisor time: 1476857258. State: :timed-out, Heartbeat: #backtype.storm.daemon.common.WorkerHeartbeat{:time-secs 1476857226, :storm-id "top3-17-1476839721", :executors #{[36 36] [6 6] [11 11] [16 16] [21 21] [26 26] [31 31] [-1 -1] [1 1]}, :port 6701}
2016-10-19T15:07:38.905+0900 b.s.d.supervisor [INFO] Shutting down b278933f-f9c7-4189-b615-1d70c7988f17:ee13ada9-641e-463a-9be5-f3ed66fdb8f3
2016-10-19T15:07:38.907+0900 b.s.util [INFO] Error when trying to kill 9306. Process is probably already dead.
2016-10-19T15:07:44.948+0900 b.s.d.supervisor [INFO] Shutting down and clearing state for id d6df820a-7c29-4bff-a606-9e8e36fafab2. Current supervisor time: 1476857264. State: :disallowed, Heartbeat: #backtype.storm.daemon.common.WorkerHeartbeat{:time-secs 1476857264, :storm-id "top3-17-1476839721", :executors #{[-1 -1]}, :port 6701}
2016-10-19T15:07:44.949+0900 b.s.d.supervisor [INFO] Shutting down b278933f-f9c7-4189-b615-1d70c7988f17:d6df820a-7c29-4bff-a606-9e8e36fafab2
2016-10-19T15:07:45.954+0900 b.s.util [INFO] Error when trying to kill 11171. Process is probably already dead.
2016-10-19T15:07:45.954+0900 b.s.d.supervisor [INFO] Shut down b278933f-f9c7-4189-b615-1d70c7988f17:d6df820a-7c29-4bff-a606-9e8e36fafab2

在zookeeper.out日志中......显示如下（xxx ip地址是另一个风暴动物园管理员地址）

2016-09-20 02:31:06,031 [myid:5] - INFO  [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxn@1007] - Closed socket connection for client /xxx.xxx.xxx.xxx:39426 which had sessionid 0x5574372bbf00004
2016-09-20 02:31:08,116 [myid:5] - WARN  [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxn@357] - caught end of stream exception
EndOfStreamException: Unable to read additional data from client sessionid 0x5574372bbf0000a, likely client has closed socket
    at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228)
    at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
    at java.lang.Thread.run(Thread.java:745)

我不知道为什么工人经常倒闭。我该如何解决？这有什么不对吗？

哦，我的动物园管理员和风暴配置如下所示

zoo.cfg（所有节点相同）

tickTime=2000
initLimit=10
syncLimit=5
dataDir=/log/zkdata/1
clientPort=2181
server.1=storm01:2888:3888
server.2=storm02:2888:3888
server.3=storm03:2888:3888
server.4=storm04:2888:3888
server.5=storm05:2888:3888
autopurge.purgeInterval=1

storm.yaml

storm.zookeeper.servers:
 - "storm01"
 - "storm02"
 - "storm03"
 - "storm04"
 - "storm05"

storm.zookeeper.port: 2181
zookeeper.multiple.setup:
 follower.port:2888
 election.port:3888

nimbus.host: "storm01"

storm.supervisor.hosts:
  - "storm01"
  - "storm02"
  - "storm03"
  - "storm04"
  - "storm05"

supervisor.slots.ports:
 - 6700
 - 6701
 - 6702
 - 6703
 - 6704

storm.local.dir: /log/storm-data

worker.childopts: "-Xmx5120m -Djava.net.preferIPv4Stack=true"

topology.workers: 5

storm.log.dir: /log/storm-log

Apache风暴主管经常关闭工作人员

0 个答案: