我写了一个简单的程序(“控制器”)来在一个单独的节点(“worker”)上运行一些计算。原因是如果工作节点内存不足,控制器仍然可以工作:
-module(controller).
-compile(export_all).
p(Msg,Args) -> io:format("~p " ++ Msg, [time() | Args]).
progress_monitor(P,N) ->
timer:sleep(5*60*1000),
p("killing the worker which was using strategy #~p~n", [N]),
exit(P, took_to_long).
start() ->
start(1).
start(Strat) ->
P = spawn('worker@localhost', worker, start, [Strat,self(),60000000000]),
p("starting worker using strategy #~p~n", [Strat]),
spawn(controller,progress_monitor,[P,Strat]),
monitor(process, P),
receive
{'DOWN', _, _, P, Info} ->
p("worker using strategy #~p died. reason: ~p~n", [Strat, Info]);
X ->
p("got result: ~p~n", [X])
end,
case Strat of
4 -> p("out of strategies. giving up~n", []);
_ -> timer:sleep(5000), % wait for node to come back
start(Strat + 1)
end.
为了测试它,我故意编写了3个使用大量内存和崩溃的析构实现,以及第四个使用尾递归来避免占用太多空间的实现:
-module(worker).
-compile(export_all).
start(1,P,N) -> P ! factorial1(N);
start(2,P,N) -> P ! factorial2(N);
start(3,P,N) -> P ! factorial3(N);
start(4,P,N) -> P ! factorial4(N,1).
factorial1(0) -> 1;
factorial1(N) -> N*factorial1(N-1).
factorial2(N) ->
case N of
0 -> 1;
_ -> N*factorial2(N-1)
end.
factorial3(N) -> lists:foldl(fun(X,Y) -> X*Y end, 1, lists:seq(1,N)).
factorial4(0, A) -> A;
factorial4(N, A) -> factorial4(N-1, A*N).
请注意,即使使用尾递归版本,我也会使用60000000000调用它,即使使用factorial4
,我的机器也可能需要几天。以下是运行控制器的输出:
$ erl -sname 'controller@localhost'
Erlang R16B (erts-5.10.1) [source] [64-bit] [smp:4:4] [async-threads:10] [hipe] [kernel-poll:false]
Eshell V5.10.1 (abort with ^G)
(controller@localhost)1> c(worker).
{ok,worker}
(controller@localhost)2> c(controller).
{ok,controller}
(controller@localhost)3> controller:start().
{23,24,28} starting worker using strategy #1
{23,25,13} worker using strategy #1 died. reason: noconnection
{23,25,18} starting worker using strategy #2
{23,26,2} worker using strategy #2 died. reason: noconnection
{23,26,7} starting worker using strategy #3
{23,26,40} worker using strategy #3 died. reason: noconnection
{23,26,45} starting worker using strategy #4
{23,29,28} killing the worker which was using strategy #1
{23,29,29} worker using strategy #4 died. reason: took_to_long
{23,29,29} out of strategies. giving up
ok
它几乎可以工作,但工人#4过早被杀(应该接近23:31:45,而不是23:29:29)。看起来更深,只有工人#1被企图杀死,而没有其他人。所以工人#4不应该死,但确实如此。为什么?我们甚至可以看到原因是took_to_long
,progress_monitor
#1从23:24:28开始,在23:29:29之前的五分钟。所以它看起来像progress_monitor
#1杀死了#4工人而不是工人#1。为什么它会扼杀错误的过程?
这是运行控制器时工人的输出:
$ while true; do erl -sname 'worker@localhost'; done
Erlang R16B (erts-5.10.1) [source] [64-bit] [smp:4:4] [async-threads:10] [hipe] [kernel-poll:false]
Eshell V5.10.1 (abort with ^G)
(worker@localhost)1>
Crash dump was written to: erl_crash.dump
eheap_alloc: Cannot allocate 2733560184 bytes of memory (of type "heap").
Aborted
Erlang R16B (erts-5.10.1) [source] [64-bit] [smp:4:4] [async-threads:10] [hipe] [kernel-poll:false]
Eshell V5.10.1 (abort with ^G)
(worker@localhost)1>
Crash dump was written to: erl_crash.dump
eheap_alloc: Cannot allocate 2733560184 bytes of memory (of type "heap").
Aborted
Erlang R16B (erts-5.10.1) [source] [64-bit] [smp:4:4] [async-threads:10] [hipe] [kernel-poll:false]
Eshell V5.10.1 (abort with ^G)
(worker@localhost)1>
Crash dump was written to: erl_crash.dump
eheap_alloc: Cannot allocate 2733560184 bytes of memory (of type "old_heap").
Aborted
Erlang R16B (erts-5.10.1) [source] [64-bit] [smp:4:4] [async-threads:10] [hipe] [kernel-poll:false]
Eshell V5.10.1 (abort with ^G)
(worker@localhost)1>
答案 0 :(得分:2)
有几个问题,最终您遇到了创建编号。
由于您未取消progress_monitor
进程,会在5分钟后发送退出信号。
计算很长和/或VM很慢,因此在进程1的进度监视器启动后5分钟,进程4仍在运行。
4个工作节点按顺序启动,名称相同workers@localhost
,第一个和第四个节点的创建编号相同。
创建编号(引用和pid中的创建字段)是一种机制,用于防止由崩溃节点创建的pid和引用由具有相同名称的新节点解释。当您在节点早已消失后尝试杀死worker 1时,您对代码的期望正是如此,您不打算在重新启动的节点中终止进程。
当节点发送pid或引用时,it encodes its creation number。当它从另一个节点收到pid或引用时,它会检查pid中的创建号是否与其自己的创建号匹配。创建编号归epmd
following the 1,2,3 sequence所有。
不幸的是,在第4个节点获得退出消息时,创建号匹配,因为此序列已包装。由于节点产生进程并在之前做了完全相同的事情(初始化的erlang),节点4的worker的pid与节点1的worker的pid匹配。
结果,控制器最终杀死了工人4,认为它是工人1。
为了避免这种情况,如果在控制器的pid或引用的生命周期内可以有4个工作者,则需要比创建号更强大的东西。