本地Storm集群设置-未执行

时间:2019-12-13 02:58:13

标签: apache apache-zookeeper apache-storm

我正在尝试在同一台机器上运行Zookeeper,nimbus和Supervisor来模拟风暴集群(我知道Storm是作为分布式系统设计的,但出于学习目的是要模拟事物在单台机器上的工作方式。

Zookeeper,Nimbus和Supervisor顺利启动且没有错误,我提交了一个风暴启动程序拓扑(RollingTopWords)。 storm.yaml的zookeeper和nimbus都指向127.0.0.1。 Nimbus获得了拓扑,并且似乎开始下载到主管,但是由于某些原因看不到主管有任何执行。除启动消息外,主管日志也为空。

雨云日志在下面(最后几个),在那之后我什么也没看见。

2019-12-13 02:43:01.818 o.a.s.d.n.Nimbus timer [INFO] Assigning top1-1-1576204977 to 1 slots 2019-12-13 02:43:01.818 o.a.s.d.n.Nimbus timer [INFO] Assign executors: [[14, 14], [12, 12], [10, 10], [8, 8], [6, 6], [4, 4], [2, 2], [7, 7], [5, 5], [3, 3], [1, 1], [15, 15], [13, 13], [11, 11], [9, 9]] 2019-12-13 02:43:01.819 o.a.s.d.n.Nimbus timer [INFO] Fragmentation after scheduling is: 0.0 MB, 0 PCore CPUs 2019-12-13 02:43:01.819 o.a.s.d.n.Nimbus timer [INFO] Node Id: 1a8a2795-2b6f-4529-862c-61c45d7bf536-10.148.0.3 Total Mem: 4096.0, Used Mem: 1920.0, Available Mem: 2176.0, Total CPU: 400.0, Used CPU: 150.0, Available CPU: 250.0, fragmented: false 2019-12-13 02:43:01.819 o.a.s.d.n.Nimbus timer [INFO] Setting new assignment for topology id top1-1-1576204977: Assignment(master_code_dir:storm-local, node_host:{1a8a2795-2b6f-4529-862c-61c45d7bf536-10.148.0.3=centos-7-1.asia-southeast1-b.c.mandava-forever.internal}, executor_node_port:{[14, 14]=NodeInfo(node:1a8a2795-2b6f-4529-862c-61c45d7bf536-10.148.0.3, port:[6700]), [12, 12]=NodeInfo(node:1a8a2795-2b6f-4529-862c-61c45d7bf536-10.148.0.3, port:[6700]), [10, 10]=NodeInfo(node:1a8a2795-2b6f-4529-862c-61c45d7bf536-10.148.0.3, port:[6700]), [8, 8]=NodeInfo(node:1a8a2795-2b6f-4529-862c-61c45d7bf536-10.148.0.3, port:[6700]), [6, 6]=NodeInfo(node:1a8a2795-2b6f-4529-862c-61c45d7bf536-10.148.0.3, port:[6700]), [4, 4]=NodeInfo(node:1a8a2795-2b6f-4529-862c-61c45d7bf536-10.148.0.3, port:[6700]), [2, 2]=NodeInfo(node:1a8a2795-2b6f-4529-862c-61c45d7bf536-10.148.0.3, port:[6700]), [7, 7]=NodeInfo(node:1a8a2795-2b6f-4529-862c-61c45d7bf536-10.148.0.3, port:[6700]), [5, 5]=NodeInfo(node:1a8a2795-2b6f-4529-862c-61c45d7bf536-10.148.0.3, port:[6700]), [3, 3]=NodeInfo(node:1a8a2795-2b6f-4529-862c-61c45d7bf536-10.148.0.3, port:[6700]), [1, 1]=NodeInfo(node:1a8a2795-2b6f-4529-862c-61c45d7bf536-10.148.0.3, port:[6700]), [15, 15]=NodeInfo(node:1a8a2795-2b6f-4529-862c-61c45d7bf536-10.148.0.3, port:[6700]), [13, 13]=NodeInfo(node:1a8a2795-2b6f-4529-862c-61c45d7bf536-10.148.0.3, port:[6700]), [11, 11]=NodeInfo(node:1a8a2795-2b6f-4529-862c-61c45d7bf536-10.148.0.3, port:[6700]), [9, 9]=NodeInfo(node:1a8a2795-2b6f-4529-862c-61c45d7bf536-10.148.0.3, port:[6700])}, executor_start_time_secs:{[2, 2]=1576204981, [4, 4]=1576204981, [6, 6]=1576204981, [8, 8]=1576204981, [10, 10]=1576204981, [12, 12]=1576204981, [14, 14]=1576204981, [1, 1]=1576204981, [3, 3]=1576204981, [5, 5]=1576204981, [7, 7]=1576204981, [9, 9]=1576204981, [11, 11]=1576204981, [13, 13]=1576204981, [15, 15]=1576204981}, worker_resources:{NodeInfo(node:1a8a2795-2b6f-4529-862c-61c45d7bf536-10.148.0.3, port:[6700])=WorkerResources(mem_on_heap:1920.0, mem_off_heap:0.0, cpu:150.0, shared_mem_on_heap:0.0, shared_mem_off_heap:0.0, resources:{onheap.memory.mb=1920.0, cpu.pcore.percent=150.0, offheap.memory.mb=0.0}, shared_resources:{})}, total_shared_off_heap:{}, owner:root) 2019-12-13 02:43:02.849 o.a.s.d.n.Nimbus pool-29-thread-41 [INFO] Created download session 2e19912c-f36f-4eb6-9b1e-df51b94adb24 for top1-1-1576204977-stormjar.jar 2019-12-13 02:43:02.851 o.a.s.d.n.Nimbus pool-29-thread-39 [INFO] Created download session 1798dab4-32d0-4112-b20e-e7b96eee4eaf for top1-1-1576204977-stormcode.ser 2019-12-13 02:43:02.859 o.a.s.d.n.Nimbus pool-29-thread-34 [INFO] Created download session b0523592-8374-43f1-b536-22c724bb8836 for top1-1-1576204977-stormconf.ser

1 个答案:

答案 0 :(得分:0)

我发现-它实际上正在运行,执行日志存储在Storm的每个工作日志中。 在我的设置中,它已登录/logs/workers-artifacts/-x-xxxx/670x/worker.log

670X是指主管上的工作人员端口号。例如worker1会有6701等。