Question

我不确定在这里我对joblib做错了，循环运行2次（相同的迭代被调用多次（两次），这是不可取的）。主函数(words_2)有很多参数，所以我使用了functools的部分参数。如果您需要words_2 function我可以发布它。它功能很长。

words_2(string, folder, fontface, fontface_italic, number_of_lines, 
       highlight, highlight_color, font_color, key_color, 
       first_key, second_key, third_key, stroke_color, 
       stroke_width, txt_under_color)

我希望在调用joblib进行一次迭代时添加的另一件事就是这样，但是我需要实现多处理。

Parallel(n_jobs=1, verbose=50)(delayed(words_2_partial)(string=sentence, folder=sf) for sentence in sep[0:1] for sf in folder[0:1])
Parallel(n_jobs=1, verbose=50)(delayed(words_2_partial)(string=sentence, folder=sf) for sentence in sep[1:2] for sf in folder[1:2])
Parallel(n_jobs=1, verbose=50)(delayed(words_2_partial)(string=sentence, folder=sf) for sentence in sep[2:3] for sf in folder[2:3])

以下是代码：

if __name__ == '__main__':

    sep = ['These limits may help reduce', 'though not completely eliminate', 'alcohol related risks']

    folder = [os.makedirs(os.path.join("my_fo", str(idx))) for idx, string in enumerate(sep) if not os.path.exists(os.path.join("my_fo", str(idx)))]
    print folder # ['my_fo\\0', 'my_fo\\1', 'my_fo\\2']

    # I am using functools's partial since only string and folder are arrays.
    words_2_partial = partial(words_2,\
                              fontface=fontface,\
                              fontface_italic=fontface_italic,\
                              number_of_lines=2,\
                              highlight=highlight,\
                              highlight_color=highlight_color,\
                              font_color=font_color,\
                              key_color=key_color,\
                              first_key=10,\
                              second_key=10,\
                              third_key=10,\
                              stroke_color=stroke_color,\
                              stroke_width=stroke_width,\
                              txt_under_color=txt_under_color
                              )


    # Now calling joblib...
    Parallel(n_jobs=len(sep), verbose=50)(delayed(words_2_partial)(string=s, folder=sf) for s in sep for sf in folder)

多处理时多次运行相同的循环/迭代

0 个答案: