Python的SyslogHandler和TCP

时间:2016-10-14 11:09:24

标签: python tcp syslog fluentd

我试图理解为什么来自Python的日志框架(logging.handlers)的SyslogHandler类没有实现RFC 6587所描述的任何框架机制:

  1. 八位字节计数:它" prepends" syslog框架的消息长度:

  2. 非透明框架:用于分隔邮件的预告片字符。这就是大多数服务器所理解的。

  3. 这个"问题"可以通过在消息的末尾添加一个LF字符来轻松解决,但是我希望SyslogHandler默认处理这个问题:

    'is admin, return true'

    这既不适用于Fluentd,也不适用于rsyslog。正如我所说,我暂时将其添加到handlers.py的第855行(仅用于测试):

    sHandler = logging.handlers.SysLogHandler(address=(address[0], address[1]), socktype = socket.SOCK_STREAM)
    sHandler.setFormatter(logging.Formatter(fmt=MSG_SYSLOG_FORMAT, datefmt=DATE_FMT))
    self.addHandler(sHandler)
    

    现在正在努力。

    我的问题:

    1. Python SyslogHandler类是否可以设置打开/关闭八位字节计数或预告字符。目前它没有任何作用......
    2. 程序员的工作是了解服务器的工作原理并覆盖处理程序以解决消息框架问题?
    3. 目前,我现在所做的是覆盖emit()方法,对SyslogHandler进行子类化。

3 个答案:

答案 0 :(得分:1)

logging中的Syslog支持早于RFC,在此之前,标准的方式很少。

准确地说:SysLogHandler处理程序在first added to the Python standard library in 2002时是logging的一部分,并且自那以后基本保持不变(TCP支持为added in 2009,并且RFC5424支持得到改进在2011年);原始代码基于this syslog module from 1997

other bug reports很明显,维护者希望在此处保持代码中最广泛的向后兼容性,因此如果您需要来自较新RFC的特定功能,则有两种选择:

  • 扩展课程并自行实施该功能
  • 提交功能请求和/或补丁以改进logging模块中的功能;考虑到向后兼容性要求。

答案 1 :(得分:1)

感谢@Martijn Pieters♦ 的回答,我的回答扩展了他的回答。

我实现了一个继承自 SyslogHandler 类并覆盖 emit 函数的类。 我还为此问题打开了一个拉取请求: https://github.com/python/cpython/pull/24556

python2:

import socket
import logging.handlers as handlers


class TcpSyslogHandler(handlers.SysLogHandler):
    """
    This class override the python SyslogHandler emit function.
    It is needed to deal with appending of the nul character to the end of the message when using TCP.
    Please see: https://stackoverflow.com/questions/40041697/pythons-sysloghandler-and-tcp/40152493#40152493
    """
    def __init__(self, message_separator_character, address=('localhost', handlers.SYSLOG_UDP_PORT),
                 facility=handlers.SysLogHandler.LOG_USER,
                 socktype=None):
        """
        The user of this class must specify the value for the messages separator.
        :param message_separator_character: The value to separate between messages.
                                            The recommended value is the "nul character": "\000".
        :param address: Same as in the super class.
        :param facility: Same as in the super class.
        :param socktype: Same as in the super class.
        """
        super(SfTcpSyslogHandler, self).__init__(address=address, facility=facility, socktype=socktype)

        self.message_separator_character = message_separator_character

    def emit(self, record):
        """
        SFTCP addition:
        To let the user to choose which message_separator_character to use, we override the emit function.
        ####
        Emit a record.

        The record is formatted, and then sent to the syslog server. If
        exception information is present, it is NOT sent to the server.
        """
        try:
            msg = self.format(record) + self.message_separator_character

            """
            We need to convert record level to lowercase, maybe this will
            change in the future.
            """
            prio = '<%d>' % self.encodePriority(self.facility, self.mapPriority(record.levelname))
            # Message is a string. Convert to bytes as required by RFC 5424
            if type(msg) is unicode:
                msg = msg.encode('utf-8')
            msg = prio + msg
            if self.unixsocket:
                try:
                    self.socket.send(msg)
                except socket.error:
                    self.socket.close()  # See issue 17981
                    self._connect_unixsocket(self.address)
                    self.socket.send(msg)
            elif self.socktype == socket.SOCK_DGRAM:
                self.socket.sendto(msg, self.address)
            else:
                self.socket.sendall(msg)
        except (KeyboardInterrupt, SystemExit):
            raise
        except Exception:
            self.handleError(record)

python3:

import socket
import logging.handlers as handlers


class SfTcpSyslogHandler(handlers.SysLogHandler):
    """
    This class override the python SyslogHandler emit function.
    It is needed to deal with appending of the nul character to the end of the message when using TCP.
    Please see: https://stackoverflow.com/questions/40041697/pythons-sysloghandler-and-tcp/40152493#40152493
    """
    def __init__(self, message_separator_character, address=('localhost', handlers.SYSLOG_UDP_PORT),
                 facility=handlers.SysLogHandler.LOG_USER,
                 socktype=None):
        """
        The user of this class must specify the value for the messages separator.
        :param message_separator_character: The value to separate between messages.
                                            The recommended value is the "nul character": "\000".
        :param address: Same as in the super class.
        :param facility: Same as in the super class.
        :param socktype: Same as in the super class.
        """
        super(SfTcpSyslogHandler, self).__init__(address=address, facility=facility, socktype=socktype)

        self.message_separator_character = message_separator_character

    def emit(self, record):
        """
        SFTCP addition:
        To let the user to choose which message_separator_character to use, we override the emit function.
        ####
        Emit a record.

        The record is formatted, and then sent to the syslog server. If
        exception information is present, it is NOT sent to the server.
        """
        try:
            msg = self.format(record) + self.message_separator_character
            if self.ident:
                msg = self.ident + msg

            # We need to convert record level to lowercase, maybe this will
            # change in the future.
            prio = '<%d>' % self.encodePriority(self.facility,
                                                self.mapPriority(record.levelname))
            prio = prio.encode('utf-8')
            # Message is a string. Convert to bytes as required by RFC 5424
            msg = msg.encode('utf-8')
            msg = prio + msg
            if self.unixsocket:
                try:
                    self.socket.send(msg)
                except OSError:
                    self.socket.close()
                    self._connect_unixsocket(self.address)
                    self.socket.send(msg)
            elif self.socktype == socket.SOCK_DGRAM:
                self.socket.sendto(msg, self.address)
            else:
                self.socket.sendall(msg)
        except Exception:
            self.handleError(record)

答案 2 :(得分:0)

如果问题被标记为fluentd,您是否尝试使用fluent.handler.FluentHandler代替logging.handlers.SysLogHandler - 请参阅https://github.com/fluent/fluent-logger-python