我应该如何编写记录器代理?

时间:2020-06-10 15:03:16

标签: python logging multiprocessing python-3.8

我具有制作记录器代理的功能,可以安全地传递给多处理工作者,并使用以下代码重新登录到主记录器:

import logging
from multiprocessing.managers import BaseManager

class SimpleGenerator:
    def __init__(self, obj): self._obj = obj
    def __call__(self): return self._obj

def get_logger_proxy(logger):
    class LoggerManager(BaseManager): pass
    logger_generator = SimpleGenerator(logger)
    LoggerManager.register('logger', callable = logger_generator)
    logger_manager = LoggerManager()
    logger_manager.start()
    logger_proxy = logger_manager.logger()

    return logger_proxy

logger = logging.getLogger('test')

logger_proxy = get_logger_proxy(logger)

这在python 2.7到3.7上效果很好。我可以将生成的logger_proxy传递给工作人员,他们将记录信息,然后将这些信息正确发送回主logger

但是,在python 3.8.2(和3.8.0)上,我得到以下信息:

Traceback (most recent call last):
  File "junk.py", line 20, in <module>
    logger_proxy = get_logger_proxy(logger)
  File "junk.py", line 13, in get_logger_proxy
    logger_manager.start()
  File "/anaconda3/envs/py3.8/lib/python3.8/multiprocessing/managers.py", line 579, in start
    self._process.start()
  File "/anaconda3/envs/py3.8/lib/python3.8/multiprocessing/process.py", line 121, in start
    self._popen = self._Popen(self)
  File "/anaconda3/envs/py3.8/lib/python3.8/multiprocessing/context.py", line 283, in _Popen
    return Popen(process_obj)
  File "/anaconda3/envs/py3.8/lib/python3.8/multiprocessing/popen_spawn_posix.py", line 32, in __init__
    super().__init__(process_obj)
  File "/anaconda3/envs/py3.8/lib/python3.8/multiprocessing/popen_fork.py", line 19, in __init__
    self._launch(process_obj)
  File "/anaconda3/envs/py3.8/lib/python3.8/multiprocessing/popen_spawn_posix.py", line 47, in _launch
    reduction.dump(process_obj, fp)
  File "/anaconda3/envs/py3.8/lib/python3.8/multiprocessing/reduction.py", line 60, in dump
    ForkingPickler(file, protocol).dump(obj)
AttributeError: Can't pickle local object 'get_logger_proxy.<locals>.LoggerManager'

因此,似乎他们更改了有关ForkingPickler的某些内容,从而使其无法处理我的get_logger_proxy函数中的关闭操作。

我的问题是,如何解决此问题,使其在python 3.8中工作?还是有一种更好的方法来获取可以在python 3.8中工作的记录器代理,就像以前版本中的记录器代理那样?

0 个答案:

没有答案