假设我有一个昂贵的函数/可迭代对,我想与multiprocessing.Pool()
的{{1}}或.map()
类方法并行化:
.map_async()
尽管我们的代码与import time
from multiprocessing import Pool
def my_expensive_function(x):
time.sleep(x)
return x*x
list_of_nums = list(range(0,100))
with Pool() as p:
nums_sq = p.map_async(func = my_expensive_function,
iterable = list_of_nums).get()
print(nums_sq)
并行化并且比我们仅使用.map_async()
循环要快得多,但我希望能够中断for
来结束子进程过早地获得部分结果。
天真的try-except块无法正常工作:
.map_async()
(值得注意的是,只要在键盘中断之前for循环至少迭代一次,就可以在for循环中使用类似的try-except块。)
因此,为了定义import time
from multiprocessing import Pool
# Now with try/except:
def my_expensive_function(x):
time.sleep(x)
return x*x
list_of_nums = list(range(0,100))
with Pool() as p:
try:
nums_sq = p.map_async(func = my_expensive_function,
iterable = list_of_nums).get()
except KeyboardInterrupt:
pass
# If interrupted before .map_async() has completed...
print(nums_sq) # NameError: name 'nums_sq2' is not defined
并指向具有nums_sq
调用的部分结果的对象,这里需要进行哪些修改?