每次我用iPython parallel (ipyparallel)
运行非常耗时的任务时,都会收到一条弹出消息"内核重启内核似乎已经死亡。它会自动重启。"
调试这种情况的好方法是什么?我在我的程序上打印了一行,可能在完成所有进程后,内核终止并重新启动。
,即,在下面的代码中,在执行run_parallel
几次(通常是10次)之后,内核在打印之前就会死掉"完成并行"信息。然后,在临终前,"完成99"每次都打印出来。
因此,我想在完成所有并行进程后从并行引擎返回时内核会死掉。但我不知道如何调试和修复它。
from IPython.parallel import Client
from IPython import parallel
from IPython.parallel import require
def run(x):
# run process once for x
...
print "finish run %d" % x
return result
def run_parallel(num):
print("run_parallel: %d times" % num)
rc = parallel.Client()
dview = rc[:]
try:
res = dview.map_async(run, range(100))
wait_watching_stdout() # Real-time output from engines in IPython parallel (http://stackoverflow.com/questions/18086299/real-time-output-from-engines-in-ipython-parallel)
print ("finish parallel")
result = res.get()
finally:
rc.close()
return result
all_result = []
for num in range(500):
r = run_parallel(num)
all_result.append(r)