如何中断multiprocessing.Pool()方法并获得部分结果?

时间:2019-03-13 20:28:50

标签: python error-handling multiprocessing python-3.7 try-except

假设我有一个昂贵的函数/可迭代对,我想与multiprocessing.Pool()的{​​{1}}或.map()类方法并行化:

.map_async()

尽管我们的代码与import time from multiprocessing import Pool def my_expensive_function(x): time.sleep(x) return x*x list_of_nums = list(range(0,100)) with Pool() as p: nums_sq = p.map_async(func = my_expensive_function, iterable = list_of_nums).get() print(nums_sq) 并行化并且比我们仅使用.map_async()循环要快得多,但我希望能够中断for来结束子进程过早地获得部分结果。

天真的try-except块无法正常工作:

.map_async()

(值得注意的是,只要在键盘中断之前for循环至少迭代一次,就可以在for循环中使用类似的try-except块。)

因此,为了定义import time from multiprocessing import Pool # Now with try/except: def my_expensive_function(x): time.sleep(x) return x*x list_of_nums = list(range(0,100)) with Pool() as p: try: nums_sq = p.map_async(func = my_expensive_function, iterable = list_of_nums).get() except KeyboardInterrupt: pass # If interrupted before .map_async() has completed... print(nums_sq) # NameError: name 'nums_sq2' is not defined 并指向具有nums_sq调用的部分结果的对象,这里需要进行哪些修改?

0 个答案:

没有答案