在Python记录器中覆盖lineno的最佳方法

时间:2015-09-07 17:46:37

标签: python logging decorator

我编写了一个装饰器,它记录用于调用特定函数或方法的参数。如下所示,除了logRecord中报告的行号是装饰器的行号而不是正在包装的func的行号之外,它运行良好:

from functools import wraps
import inspect
import logging

arg_log_fmt = "{name}({arg_str})"


def log_args(logger, level=logging.DEBUG):
    """Decorator to log arguments passed to func."""
    def inner_func(func):
        line_no = inspect.getsourcelines(func)[-1]

        @wraps(func)
        def return_func(*args, **kwargs):
            arg_list = list("{!r}".format(arg) for arg in args)
            arg_list.extend("{}={!r}".format(key, val)
                            for key, val in kwargs.iteritems())
            msg = arg_log_fmt.format(name=func.__name__,
                                     arg_str=", ".join(arg_list))
            logger.log(level, msg)
            return func(*args, **kwargs)
        return return_func

    return inner_func

if __name__ == "__main__":
    logger = logging.getLogger(__name__)
    handler = logging.StreamHandler()
    fmt = "%(asctime)s %(levelname)-8.8s [%(name)s:%(lineno)4s] %(message)s"
    handler.setFormatter(logging.Formatter(fmt))
    logger.addHandler(handler)
    logger.setLevel(logging.DEBUG)


    @log_args(logger)
    def foo(x, y, z):
        pass

    class Bar(object):
        @log_args(logger)
        def baz(self, a, b, c):
            pass

    foo(1, 2, z=3)
    foo(1, 2, 3)
    foo(x=1, y=2, z=3)

    bar = Bar()
    bar.baz(1, c=3, b=2)

此示例产生以下输出

2015-09-07 12:42:47,779 DEBUG    [__main__:  25] foo(1, 2, z=3)
2015-09-07 12:42:47,779 DEBUG    [__main__:  25] foo(1, 2, 3)
2015-09-07 12:42:47,779 DEBUG    [__main__:  25] foo(y=2, x=1, z=3)
2015-09-07 12:42:47,779 DEBUG    [__main__:  25] baz(<__main__.Bar object at 0x1029094d0>, 1, c=3, b=2)

请注意,行号都指向装饰器。

使用inspect.getsourcelines(func)我可以获得我感兴趣的行号,但尝试在lineno中覆盖logger.debug会导致错误。获取包装函数的行号以显示在日志记录语句中的最佳方法是什么?

4 个答案:

答案 0 :(得分:4)

另一种可能性是将Logger子类化为覆盖Logger.makeRecord。如果您尝试更改KeyError中的任何标准属性(例如rv.lineno),This is the method会引发LogRecord

for key in extra:
    if (key in ["message", "asctime"]) or (key in rv.__dict__):
        raise KeyError("Attempt to overwrite %r in LogRecord" % key)
    rv.__dict__[key] = extra[key]

通过删除此预防措施,我们可以通过提供一个来覆盖lineno值 extra来电的logger.log参数:

logger.log(level, msg, extra=dict(lineno=line_no))
from functools import wraps
import inspect
import logging

arg_log_fmt = "{name}({arg_str})"


def makeRecord(self, name, level, fn, lno, msg, args, exc_info, func=None, extra=None):
    """
    A factory method which can be overridden in subclasses to create
    specialized LogRecords.
    """
    rv = logging.LogRecord(name, level, fn, lno, msg, args, exc_info, func)
    if extra is not None:
        rv.__dict__.update(extra)
    return rv

def log_args(logger, level=logging.DEBUG, cache=dict()):
    """Decorator to log arguments passed to func."""
    logger_class = logger.__class__
    if logger_class in cache:
        UpdateableLogger = cache[logger_class]
    else:
        cache[logger_class] = UpdateableLogger = type(
            'UpdateableLogger', (logger_class,), dict(makeRecord=makeRecord))

    def inner_func(func):
        line_no = inspect.getsourcelines(func)[-1]
        @wraps(func)
        def return_func(*args, **kwargs):
            arg_list = list("{!r}".format(arg) for arg in args)
            arg_list.extend("{}={!r}".format(key, val)
                            for key, val in kwargs.iteritems())
            msg = arg_log_fmt.format(name=func.__name__,
                                     arg_str=", ".join(arg_list))
            logger.__class__ = UpdateableLogger
            try:
                logger.log(level, msg, extra=dict(lineno=line_no))
            finally:
                logger.__class__ = logger_class
            return func(*args, **kwargs)
        return return_func

    return inner_func

if __name__ == "__main__":
    logger = logging.getLogger(__name__)
    handler = logging.StreamHandler()
    fmt = "%(asctime)s %(levelname)-8.8s [%(name)s:%(lineno)4s] %(message)s"
    handler.setFormatter(logging.Formatter(fmt))
    logger.addHandler(handler)
    logger.setLevel(logging.DEBUG)

    @log_args(logger)
    def foo(x, y, z):
        pass

    class Bar(object):

        @log_args(logger)
        def baz(self, a, b, c):
            pass

    foo(1, 2, z=3)
    foo(1, 2, 3)
    foo(x=1, y=2, z=3)

    bar = Bar()
    bar.baz(1, c=3, b=2)

产量

2015-09-07 16:01:22,332 DEBUG    [__main__:  48] foo(1, 2, z=3)
2015-09-07 16:01:22,332 DEBUG    [__main__:  48] foo(1, 2, 3)
2015-09-07 16:01:22,332 DEBUG    [__main__:  48] foo(y=2, x=1, z=3)
2015-09-07 16:01:22,332 DEBUG    [__main__:  53] baz(<__main__.Bar object at 0x7f17f75b0490>, 1, c=3, b=2)

该行

    UpdateableLogger = type('UpdateableLogger', (type(logger),), 
                            dict(makeRecord=makeRecord))

创建一个新类,它是type(logger)的子类,它会覆盖makeRecord。 在return_func内,logger的类已更改为UpdateableLogger,因此对logger.log的调用可以修改lineno,然后恢复原始记录器类。

通过这种方式 - 通过避免猴子修补Logger.makeRecord - 所有logger的行为与装饰函数之外的行为完全相同。

为了进行比较,猴子修补方法是shown here

答案 1 :(得分:2)

Martijn指出,事情有时会发生变化。但是,由于您正在使用Python 2(iteritems将其删除),如果您不介意修补猴子修补程序,以下代码将起作用:

from functools import wraps
import logging

class ArgLogger(object):
    """
    Singleton class -- will only be instantiated once
    because of the monkey-patching of logger.
    """

    singleton = None

    def __new__(cls):
        self = cls.singleton
        if self is not None:
            return self
        self = cls.singleton = super(ArgLogger, cls).__new__(cls)
        self.code_location = None

        # Do the monkey patch exactly one time
        def findCaller(log_self):
            self.code_location, code_location = None, self.code_location
            if code_location is not None:
                return code_location
            return old_findCaller(log_self)
        old_findCaller = logging.Logger.findCaller
        logging.Logger.findCaller = findCaller

        return self

    def log_args(self, logger, level=logging.DEBUG):
        """Decorator to log arguments passed to func."""
        def inner_func(func):
            co = func.__code__
            code_loc = (co.co_filename, co.co_firstlineno, co.co_name)

            @wraps(func)
            def return_func(*args, **kwargs):
                arg_list = list("{!r}".format(arg) for arg in args)
                arg_list.extend("{}={!r}".format(key, val)
                                for key, val in kwargs.iteritems())
                msg = "{name}({arg_str})".format(name=func.__name__,
                                        arg_str=", ".join(arg_list))
                self.code_location = code_loc
                logger.log(level, msg)
                return func(*args, **kwargs)
            return return_func

        return inner_func


log_args = ArgLogger().log_args

if __name__ == "__main__":
    logger = logging.getLogger(__name__)
    handler = logging.StreamHandler()
    fmt = "%(asctime)s %(levelname)-8.8s [%(name)s:%(lineno)4s] %(message)s"
    handler.setFormatter(logging.Formatter(fmt))
    logger.addHandler(handler)
    logger.setLevel(logging.DEBUG)


    @log_args(logger)
    def foo(x, y, z):
        pass

    class Bar(object):
        @log_args(logger)
        def baz(self, a, b, c):
            pass

    def test_regular_log():
        logger.debug("Logging without ArgLog still works fine")

    foo(1, 2, z=3)
    foo(1, 2, 3)
    foo(x=1, y=2, z=3)

    bar = Bar()
    bar.baz(1, c=3, b=2)
    test_regular_log()

答案 2 :(得分:1)

您无法轻易更改行号,因为Logger.findCaller() method会通过内省提取此信息。

可以为你生成的包装函数重新构建函数和代码对象,但这确实非常毛茸茸(参见我和Veedrac在this post上跳过的箍)和当你遇到错误时,导致问题,因为你的追溯会显示错误的源代码行!

您最好手动添加行号以及模块名称(因为这也可能有所不同):

Prelude> :t e 1 2 3 4 5
e 1 2 3 4 5
  :: (Num ((a -> a1 -> t) -> (a -> a1 -> t) -> a -> a1 -> t),
      Num (a -> a1 -> t), Num a1, Num a) =>
     t

由于你总是在这里有一个函数,我使用了一些更直接的内省来获取函数的第一个行号,通过相关的代码对象。

答案 3 :(得分:1)

这是旧文章,但是此答案可能对其他人仍然有用。

现有解决方案的一个问题是有multiple parameters providing logging context,如果要支持任意的日志记录格式化程序,都需要修补所有这些问题。

事实证明,这是raised as an issue with the Python logging library about a year ago,结果是the stacklevel keyword argument was added in Python 3.8。使用该功能,您只需修改日志记录调用即可将堆栈级别设置为2(在示例中调用logger.log的级别之上):

logger.log(level, msg, stacklevel=2)

由于Python 3.8尚未发布(在此回复时),因此您可以使用findCaller and _log methods updated in Python 3.8对记录器进行猴子补丁。

我有一个名为logquacious的日志记录实用程序库,在该库中我进行了相同的猴子修补。您可以重复使用patch_logger class that I've defined in logquacious并使用以下命令更新上面的日志记录示例:

from logquacious.backport_configurable_stacklevel import patch_logger

logger = logging.getLogger(__name__)
logger.__class__ = patch_logger(logger.__class__)

如unutbu的答案中所述,最好在使用范围之外撤消此猴子补丁,这是该文件中其他一些代码的作用。