Question

我需要创建一个系统，在该系统中，嵌入式系统中生成的日志消息将远程记录在服务器上，并存储在轮换的日志文件中。由于网络通信的限制，必须通过HTTP协议传输日志消息。该服务器已经在运行基于Flask的HTTP服务器，因此我想将日志记录系统与基于Flask的Web应用程序集成。

Python logging模块为此提供了专用的HTTPhandler类。基于logging文档，我创建了第一个实现。

服务器部分：

from flask import Flask
from flask import Response
from flask import request
import logging
import logging.handlers
app = Flask(__name__)

@app.route('/')
def hello_world():
    return 'logging server'

@app.route('/log', methods=['POST'])
def handle_log():
    if request.method == 'POST':
       rd=request.form.to_dict()
       record = logging.makeLogRecord(rd)
       log1.handle(record)
       return "OK"

log1=logging.getLogger('MTEST')
log1.setLevel(logging.DEBUG)
fh=logging.handlers.RotatingFileHandler('logs','a',maxBytes=10000000,backupCount=10)
formatter = logging.Formatter('%(asctime)s %(name)-15s %(levelname)-8s %(message)s')
rfh.setFormatter(formatter)
log1.addHandler(rfh)
log1.error("First error generated locally")
app.run()

客户端部分：

import logging, logging.handlers
myLogger = logging.getLogger('MTEST')
myLogger.setLevel(logging.DEBUG)
httpHandler = logging.handlers.HTTPHandler('localhost:5000',url='/log',method="POST")
myLogger.addHandler(httpHandler)

myLogger.info('Small info message')
myLogger.debug('Small debug message')
myLogger.erro('Small error message')

但是，在该实现中，服务器报告了以下错误（我复制了四个几乎相同的错误消息之一），并且仅记录了本地生成的错误消息。

Traceback (most recent call last):
  File "/usr/lib/python2.7/logging/handlers.py", line 76, in emit
    if self.shouldRollover(record):
  File "/usr/lib/python2.7/logging/handlers.py", line 156, in shouldRollover
    msg = "%s\n" % self.format(record)
  File "/usr/lib/python2.7/logging/__init__.py", line 741, in format
    return fmt.format(record)
  File "/usr/lib/python2.7/logging/__init__.py", line 465, in format
    record.message = record.getMessage()
  File "/usr/lib/python2.7/logging/__init__.py", line 329, in getMessage
    msg = msg % self.args
TypeError: not all arguments converted during string formatting

我添加了打印传输到服务器的表单数据和创建的LogRecord对象的属性的信息。看来args属性是作为空的tupple传输的，而不是创建的。为了解决这个问题，我修改了handle_log例程：

@app.route('/log', methods=['POST'])
def handle_log():
    if request.method == 'POST':
       rd=request.form.to_dict()
       rd['args']=""
       record = logging.makeLogRecord(rd)
       log1.handle(record)
       return "OK"

日志仍然无法正常运行，但是服务器产生的错误已更改：

Logged from file test1.py, line 9
127.0.0.1 - - [10/Jul/2018 23:52:51] "POST /log HTTP/1.0" 200 -
Traceback (most recent call last):
  File "/usr/lib/python2.7/logging/handlers.py", line 76, in emit
    if self.shouldRollover(record):
  File "/usr/lib/python2.7/logging/handlers.py", line 156, in shouldRollover
    msg = "%s\n" % self.format(record)
  File "/usr/lib/python2.7/logging/__init__.py", line 741, in format
    return fmt.format(record)
  File "/usr/lib/python2.7/logging/__init__.py", line 467, in format
    record.asctime = self.formatTime(record, self.datefmt)
  File "/usr/lib/python2.7/logging/__init__.py", line 423, in formatTime
    ct = self.converter(record.created)
TypeError: a float is required

要检查什么地方不对，我添加了打印创建的LogRecord对象的过滤器：

from flask import Flask
from flask import Response
from flask import request
import logging
import logging.handlers
app = Flask(__name__)

@app.route('/')
def hello_world():
    return 'logging server'

@app.route('/log', methods=['POST'])
def handle_log():
    if request.method == 'POST':
       rd=request.form.to_dict()
       rd['args']=""
       record = logging.makeLogRecord(rd)
       log1.handle(record)
       return "OK"

class myflt(logging.Filter):
  def filter(self,rec):
    print(rec.__dict__)
    return 1

log1=logging.getLogger('MTEST')
log1.setLevel(logging.DEBUG)
rfh=logging.handlers.RotatingFileHandler('logs','a',maxBytes=10000000,backupCount=10)
formatter = logging.Formatter('%(asctime)s %(name)-15s %(levelname)-8s %(message)s')
rfh.setFormatter(formatter)
log1.addHandler(rfh)
log1.addFilter(myflt())
log1.error("First error generated locally")
app.run()

之后，我可以比较为本地生成的消息创建的LogRecord对象：

{'threadName': 'MainThread', 'name': 'MTEST', 'thread': 139678016341824, 'created': 1531259894.112536, 'process': 5595, 'processName': 'MainProcess', 'args': (), 'module': 'srv1', 'filename': 'srv1.py', 'levelno': 40, 'exc_text': None, 'pathname': 'srv1.py', 'lineno': 37, 'msg': 'First error generated locally', 'exc_info': None, 'funcName': '<module>', 'relativeCreated': 44.27504539489746, 'levelname': 'ERROR', 'msecs': 112.53595352172852}

对于远程生成的消息：

{'relativeCreated': u'12.3879909515', 'process': u'5597', 'module': u'test1', 'funcName': u'<module>', 'filename': u'test1.py', 'levelno': u'10', 'processName': u'MainProcess', 'lineno': u'9', 'msg': u'Small debug message', 'args': '', 'exc_text': u'None', 'name': u'MTEST', 'thread': u'140221336438592', 'created': u'1531259898.51', 'threadName': u'MainThread', 'msecs': u'511.312007904', 'pathname': u'test1.py', 'exc_info': u'None', 'levelname': u'DEBUG'}

看来，通过HTTPHandler传输日志记录会将所有数字都转换为字符串，因此makeLogRecord函数无法正确地在服务器中重新创建LogRecord对象

通过HTTPHandler传输日志消息以使它们可以由远程机器中的RotatingFileHandler正确处理的正确方法是什么？

Answer 1

我发现了一些似乎可行的解决方法。在客户端中创建的带有LogRecord对象属性的字典在传递到服务器之前会转换为JSON。然后，服务器对其进行解码，并重新创建原始的LogRecord。

服务器部分：

from flask import Flask
from flask import Response
from flask import request
import logging
import logging.handlers
import json
app = Flask(__name__)

@app.route('/')
def hello_world():
    return 'logging server'

@app.route('/log', methods=['POST'])
def handle_log():
    if request.method == 'POST':
       rd=request.form.to_dict()
       rec=json.loads(rd['record'])
       record = logging.makeLogRecord(rec)
       log1.handle(record)
       return "OK"

class myflt(logging.Filter):
  def filter(self,rec):
    print(rec.__dict__)
    return 1

mfl=myflt()

log1=logging.getLogger('MTEST')
log1.setLevel(logging.DEBUG)
rfh=logging.handlers.RotatingFileHandler('logs','a',maxBytes=10000000,backupCount=10)
formatter = logging.Formatter('%(asctime)s %(name)-15s %(levelname)-8s %(message)s')
rfh.setFormatter(formatter)
log1.addHandler(rfh)
log1.addFilter(mfl)
log1.error("First error generated locally")
app.run()

客户端部分：

import logging, logging.handlers
import json

class myHTTPHandler(logging.handlers.HTTPHandler):
  def mapLogRecord(self,record):
    trec={'record':json.dumps(record.__dict__)}
    return trec

myLogger = logging.getLogger('MTEST')
myLogger.setLevel(logging.DEBUG)
httpHandler = myHTTPHandler('localhost:5000',url='/log',method="POST")
myLogger.addHandler(httpHandler)

myLogger.info('Small info message')
myLogger.debug('Small debug message')
myLogger.error('Small error message')

上面的实现正常工作，但是似乎不必要地复杂。有没有更简单的方法可以使用HTTPHandler通过HTTP协议与远程RotatingFileHandler进行通信？

如何在Python中将HTTPHandler与RotatingFileHandler链接？

1 个答案: