使用可运行图形保存功能的pathos多处理程序遇到了问题。
from pathos.multiprocessing import ProcessingPool as Pool
datasets = Pool().map(model_handler, analysisParams)
抛出
File ".../lib/python3.6/site-packages/matplotlib/backends/backend_pdf.py", line 2029, in draw_text
font.set_text(s, 0.0, flags=LOAD_NO_HINTING)
RuntimeError: In set_text: could not load glyph
"""
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "wrapper.py", line 410, in <module>
datasets = Pool().map(model_handler, analysisParams)
File ".../lib/python3.6/site-packages/pathos/multiprocessing.py", line 137, in map
return _pool.map(star(f), zip(*args)) # chunksize
File ".../lib/python3.6/site-packages/multiprocess/pool.py", line 260, in map
return self._map_async(func, iterable, mapstar, chunksize).get()
File ".../lib/python3.6/site-packages/multiprocess/pool.py", line 608, in get
raise self._value
RuntimeError: In set_text: could not load glyph
"""
我目前的理解是,发生这种情况是因为多个进程同时调用了字形。只有在并行执行的次数大于3时才会发生例外,这与该想法是一致的。我看到one person's work around只是重复调用savefig直到命令通过为止,但是并行化函数包含大量图表,我宁愿不需要将每个包裹都放在try语句中。有谁知道在并行绘制图形时如何避免这种异常?谢谢!
编辑 每个评论请求的最小示例。
from pathos.multiprocessing import ProcessingPool as Pool
def model_handler(analysisParam):
import matplotlib
matplotlib.use('agg')
import matplotlib.pyplot as plt
figout = plt.figure(figsize=(1, 1))
axes = figout.add_subplot(1,1,1)
axes.scatter(range(1000), range(1000))
figout.savefig('dummyfig{}.pdf'.format(analysisParam), format='pdf')
return analysisParam
datasets_serial = []
for j in range(20):
datasets_serial.append(model_handler(j))
datasets = Pool(20).map(model_handler, range(20))
有趣的是,如果我用dataset_serial = map(model_handler, range(20))
替换dataset_serial循环,那么pathos并行池执行就没有问题,但是如果我将串行映射对象转换为列表(print(list(dataset_serial))
),则字形异常并行池又回来了。