Python日志记录 - 不同的日志但输出相同

时间:2012-10-26 15:31:57

标签: python parsing logging lxml

我为不同的网站提供了几种不同的解析器,我还有一个名为 shared.py 的文件,它具有用于特殊解析的 lxml 函数和一个基础.py 文件负责数据库的保存(保存等)。 如果出现故障(没有找到图像)或通过(我们发现图像),我需要将其记录到日志文件中,为此我使用的是标准日志记录模块。

我写了一个 logger.py 文件,我创建了一个Log类,所以我可以在我的解析器或 base.py/shared.py 中调用它,该文件看起来像这样:

import logging
import os.path

__metaclass__ = type

class Log:
    def __init__(self, filename, loggername="fetchers", path="logs/", formater="%(asctime)s %(levelname)s %(message)s"):
        self.logger = logging.getLogger(loggername)
        self.hdlr = logging.FileHandler(path + os.path.splitext(filename)[0] + ".log")
        self.formater = logging.Formatter(formater)
        self.hdlr.setFormatter(self.formater)
        self.logger.addHandler(self.hdlr)
        self.logger.setLevel(logging.INFO)
    def info(self, msg):
        self.logger.info(msg)
    def warning(self, msg):
        self.logger.warning(msg)
    def error(self, msg):
        self.logger.error(msg)
    def critical(self, msg):
        self.logger.critical(msg)

if __name__ == "__main__":
    pass

文件夹层次结构如下所示;

  • /解析器
  • /Parsers/base.py
  • /Parsers/shared.py
  • /Parsers/parserone.py
  • /Parsers/parsertwo.py
  • /Parsers/parsersthree.py
  • /解析器/...
  • /Parsers/logs/base.log
  • /Parsers/logs/shared.log
  • /Parsers.logs/parserOne.log

每个解析器都会导入记录器文件(也是base.py和shared.py),并且他们会像这样初始化记录器:

logger = Log(os.path.basename(__file__))
logger.warning("Something has happened..")

这样工作正常,它会写入日志,但问题是,base.py写入有关查询等的日志,以及有关失败等的解析器(与shared.py相同) 问题是它正在编写日志,但它们看起来都一样..

➜  logs  tail parserOne.log 
2012-10-26 16:35:21,250 INFO Data was sucessfuly saved: http://www.adformatie.nl/nieuws/bericht/scherpe-omzetdaling-televisiereclame/
2012-10-26 16:35:21,286 INFO Data was sucessfuly saved: http://www.adformatie.nl/nieuws/bericht/nominaties-mercurs-bekend2/
2012-10-26 16:35:21,322 INFO Data was sucessfuly saved: http://www.adformatie.nl/nieuws/bericht/traditionele-media-nog-steeds-populair-bij-jongeren/
2012-10-26 16:35:21,371 INFO Data was sucessfuly saved: http://www.adformatie.nl/nieuws/bericht/persgroep-blijft-sponsor-san/
2012-10-26 16:35:21,407 INFO Data was sucessfuly saved: http://www.adformatie.nl/nieuws/bericht/pg-overtreft-verwachtingen/
2012-10-26 16:35:21,443 INFO Data was sucessfuly saved: http://www.adformatie.nl/nieuws/bericht/discovery-networks-introduceert-discovery-client-productions/
2012-10-26 16:35:21,479 INFO Data was sucessfuly saved: http://www.adformatie.nl/nieuws/bericht/stoelendans-bij-wehkamp.nl/
2012-10-26 16:35:21,563 INFO Data was sucessfuly saved: http://www.adformatie.nl/nieuws/bericht/amazon-duikt-in-rode-cijfers/
2012-10-26 16:35:21,599 INFO Data was sucessfuly saved: http://www.adformatie.nl/nieuws/bericht/linkedin-rabobank-meest-populaire-werkgever/
2012-10-26 16:35:21,683 INFO Data was sucessfuly saved: http://www.adformatie.nl/nieuws/bericht/veronica-uitgeverij-wil-naar-amsterdam/

➜  logs  tail base.log 
2012-10-26 16:35:21,250 INFO Data was sucessfuly saved: http://www.adformatie.nl/nieuws/bericht/scherpe-omzetdaling-televisiereclame/
2012-10-26 16:35:21,286 INFO Data was sucessfuly saved: http://www.adformatie.nl/nieuws/bericht/nominaties-mercurs-bekend2/
2012-10-26 16:35:21,322 INFO Data was sucessfuly saved: http://www.adformatie.nl/nieuws/bericht/traditionele-media-nog-steeds-populair-bij-jongeren/
2012-10-26 16:35:21,371 INFO Data was sucessfuly saved: http://www.adformatie.nl/nieuws/bericht/persgroep-blijft-sponsor-san/
2012-10-26 16:35:21,407 INFO Data was sucessfuly saved: http://www.adformatie.nl/nieuws/bericht/pg-overtreft-verwachtingen/
2012-10-26 16:35:21,443 INFO Data was sucessfuly saved: http://www.adformatie.nl/nieuws/bericht/discovery-networks-introduceert-discovery-client-productions/
2012-10-26 16:35:21,479 INFO Data was sucessfuly saved: http://www.adformatie.nl/nieuws/bericht/stoelendans-bij-wehkamp.nl/
2012-10-26 16:35:21,563 INFO Data was sucessfuly saved: http://www.adformatie.nl/nieuws/bericht/amazon-duikt-in-rode-cijfers/
2012-10-26 16:35:21,599 INFO Data was sucessfuly saved: http://www.adformatie.nl/nieuws/bericht/linkedin-rabobank-meest-populaire-werkgever/
2012-10-26 16:35:21,683 INFO Data was sucessfuly saved: http://www.adformatie.nl/nieuws/bericht/veronica-uitgeverij-wil-naar-amsterdam/

➜  logs  tail shared.log 
2012-10-26 16:35:21,250 INFO Data was sucessfuly saved: http://www.adformatie.nl/nieuws/bericht/scherpe-omzetdaling-televisiereclame/
2012-10-26 16:35:21,286 INFO Data was sucessfuly saved: http://www.adformatie.nl/nieuws/bericht/nominaties-mercurs-bekend2/
2012-10-26 16:35:21,322 INFO Data was sucessfuly saved: http://www.adformatie.nl/nieuws/bericht/traditionele-media-nog-steeds-populair-bij-jongeren/
2012-10-26 16:35:21,371 INFO Data was sucessfuly saved: http://www.adformatie.nl/nieuws/bericht/persgroep-blijft-sponsor-san/
2012-10-26 16:35:21,407 INFO Data was sucessfuly saved: http://www.adformatie.nl/nieuws/bericht/pg-overtreft-verwachtingen/
2012-10-26 16:35:21,443 INFO Data was sucessfuly saved: http://www.adformatie.nl/nieuws/bericht/discovery-networks-introduceert-discovery-client-productions/
2012-10-26 16:35:21,479 INFO Data was sucessfuly saved: http://www.adformatie.nl/nieuws/bericht/stoelendans-bij-wehkamp.nl/
2012-10-26 16:35:21,563 INFO Data was sucessfuly saved: http://www.adformatie.nl/nieuws/bericht/amazon-duikt-in-rode-cijfers/
2012-10-26 16:35:21,599 INFO Data was sucessfuly saved: http://www.adformatie.nl/nieuws/bericht/linkedin-rabobank-meest-populaire-werkgever/
2012-10-26 16:35:21,683 INFO Data was sucessfuly saved: http://www.adformatie.nl/nieuws/bericht/veronica-uitgeverij-wil-naar-amsterdam/

为什么所有日志文件都一样?如果它们是不同的文件?!

干杯。

2 个答案:

答案 0 :(得分:1)

您的记录器都称为“取件器”(因为您没有给出loggername参数),因此您将在所有日志中显示相同的消息。

我建议为记录器添加名称,也许使用filters

您可以将此应用于您的代码,如下所示:

logger = Log(os.path.basename(__file__), loggername='test')

def __init__(self, filename, loggername="fetchers", path="logs/", formater="%(asctime)s %(levelname)s %(message)s"):
    self.logger = logging.getLogger(loggername)
    self.hdlr = logging.FileHandler(path + os.path.splitext(filename)[0] + ".log")
    self.hdlr.addFilter(logging.Filter(name=loggername))
    ...

答案 1 :(得分:1)

看起来它使用loggerName获取记录器,并始终设置为"fetchers"。所以你到处都使用相同的记录器,这解释了为什么输出是相同的。