执行造纸厂时内核死亡

时间:2020-03-03 21:49:56

标签: python jupyter-notebook jupyter papermill

我正在尝试使用以下代码自动使用造纸厂生成多个笔记本:

template = "preprocessing_template.ipynb"
parameters = {"allowed_values":[0,1], "epsilon":0.01, "to_csv":True}
kernel_name = "my_env"
grid_folders_csv_names_outputs = [("dir_a", "a.csv","output_a.ipynb"),\
                                  ("dir_b", "b.csv","output_b.ipynb"),\
                                  ("dir_c", "c.csv","output_c.ipynb"),
                                 [

def execute_notebook(template,output_file,parameters,kernel_name,grid_folder,csv_name):

    parameters["grid_folder"]=grid_folder
    parameters["csv_name"]=csv_name
    nb = pm.execute_notebook(template,
                        output_file,
                        parameters=parameters,
                        kernel_name=kernel_name)
    #nbs = []
    #nbs.append(nb)
    return nb

for grid_folder, csv_name, output_file in tqdm(grid_folders_csv_names_outputs):
    execute_notebook(template,output_file,parameters,kernel_name,grid_folder,csv_name)

尽管如此,我仍然收到错误消息

DeadKernelError: Kernel died

这似乎是因为我在运行代码时通过监视系统进行了检查,因此内存完全被淹没了。显然,上面的代码片段生成了多个进程来运行带参数的笔记本,并且一旦迭代完成,这些进程就不会终止,因此会在内存中累积。我该怎么做才能解决这个问题?我正在使用Ubuntu 18.04,并且代码在anaconda 2019.10环境中运行。

0 个答案:

没有答案