为什么TimedRotatingFileHandler不删除旧文件?

时间:2019-03-14 19:12:18

标签: python python-3.x logging

我正在使用TimedRotatingFileHandler创建日志。 我希望每分钟创建一次日志文件,最多保留2个日志文件并删除较旧的日志文件。这是示例代码:

import logging
import logging.handlers
import datetime

logger = logging.getLogger('MyLogger')
logger.setLevel(logging.DEBUG)

handler = logging.handlers.TimedRotatingFileHandler(
    "logs/{:%H-%M}.log".format(datetime.datetime.now()), 
    when="M", 
    backupCount=2)

logger.addHandler(handler)
logger.debug("PLEASE DELETE PREVIOUS FILES")

如果我多次运行此代码(间隔一分钟),我的日志目录中会出现多个文件,如下所示:

21-01.log
21-02.log
21-03.log
...

这对我来说似乎很奇怪,因为我设置了backupCount=2,这表示最多应保存2个文件,而较旧的文件应删除。但是,当我在日志文件夹中使用2个或更多文件启动应用程序时,不会删除旧文件。

为什么TimedRotatingFileHandler不删除旧文件? 有什么方法可以设置TimedRotatingFileHandler删除旧文件?

3 个答案:

答案 0 :(得分:2)

按设计用途,您不能使用TimedRotatingFileHandler。处理程序希望“当前”日志文件名保持稳定,并定义轮换为通过 renaming 将现有日志文件移动到备份中。这些是保留或删除的备份。轮换备份是从基本文件名加上带有轮换时间戳的后缀创建的。因此,该实现区分了日志文件(存储在baseFilename中和循环文件(生成在doRotate() method中。请注意,只有在发生旋转时才删除备份,因此在处理程序用于至少一个完整的时间间隔。

相反,您希望基本文件名本身包含时间信息,因此请更改 log文件名本身。在这种情况下,没有“备份”,您只需在轮播时刻打开一个新文件。此外,您似乎正在运行短命 Python代码,因此您希望立即删除较旧的文件,而不仅仅是在明确旋转时删除,而这可能是永远无法访问的。

这就是TimedRotatingFileHandler不会删除任何文件的原因,因为*它永远无法创建备份文件。没有备份意味着没有要删除的备份。要旋转文件,处理程序的当前实现应负责文件名的生成,并且不能期望知道它本身不会生成的文件名。当您以每分钟"M"的旋转频率对其进行配置时,它被配置为将文件旋转为具有模式{baseFileame}.{now:%Y-%m-%d_%H_%M}的备份文件,因此将仅删除与该模式匹配的旋转备份文件。参见documentation

  

系统将通过在文件名后附加扩展名来保存旧的日志文件。这些扩展基于日期和时间,使用strftime格式%Y-%m-%d_%H-%M-%S或其前导部分,具体取决于过渡间隔。

相反,您需要的是一个基本文件名,该文件名本身带有时间戳,并且在打开新日志文件时使用的名称与删除旧日志文件(而不是备份文件)的名称不同。为此,您必须创建一个自定义处理程序。

幸运的是,类层次结构是专为轻松定制而设计的。您可以在此处创建BaseRotatingHandler的子类,并提供自己的删除逻辑:

import os
import time
from itertools import islice
from logging.handlers import BaseRotatingHandler, TimedRotatingFileHandler

# rotation intervals in seconds
_intervals = {
    "S": 1,
    "M": 60,
    "H": 60 * 60,
    "D": 60 * 60 * 24,
    "MIDNIGHT": 60 * 60 * 24,
    "W": 60 * 60 * 24 * 7,
}

class TimedPatternFileHandler(BaseRotatingHandler):
    """File handler that uses the current time in the log filename.

    The time is quantisized to a configured interval. See
    TimedRotatingFileHandler for the meaning of the when, interval, utc and
    atTime arguments.

    If backupCount is non-zero, then older filenames that match the base
    filename are deleted to only leave the backupCount most recent copies,
    whenever opening a new log file with a different name.

    """

    def __init__(
        self,
        filenamePattern,
        when="h",
        interval=1,
        backupCount=0,
        encoding=None,
        delay=False,
        utc=False,
        atTime=None,
    ):
        self.when = when.upper()
        self.backupCount = backupCount
        self.utc = utc
        self.atTime = atTime
        try:
            key = "W" if self.when.startswith("W") else self.when
            self.interval = _intervals[key]
        except KeyError:
            raise ValueError(
                f"Invalid rollover interval specified: {self.when}"
            ) from None
        if self.when.startswith("W"):
            if len(self.when) != 2:
                raise ValueError(
                    "You must specify a day for weekly rollover from 0 to 6 "
                    f"(0 is Monday): {self.when}"
                )
            if not "0" <= self.when[1] <= "6":
                raise ValueError(
                    f"Invalid day specified for weekly rollover: {self.when}"
                )
            self.dayOfWeek = int(self.when[1])

        self.interval = self.interval * interval
        self.pattern = os.path.abspath(os.fspath(filenamePattern))

        # determine best time to base our rollover times on
        # prefer the creation time of the most recently created log file.
        t = now = time.time()
        entry = next(self._matching_files(), None)
        if entry is not None:
            t = entry.stat().st_ctime
            while t + self.interval < now:
                t += self.interval

        self.rolloverAt = self.computeRollover(t)

        # delete older files on startup and not delaying
        if not delay and backupCount > 0:
            keep = backupCount
            if os.path.exists(self.baseFilename):
                keep += 1
                delete = islice(self._matching_files(), keep, None)
                for entry in delete:
                    os.remove(entry.path)

        # Will set self.baseFilename indirectly, and then may use
        # self.baseFilename to open. So by this point self.rolloverAt and
        # self.interval must be known.
        super().__init__(filenamePattern, "a", encoding, delay)

    @property
    def baseFilename(self):
        """Generate the 'current' filename to open"""
        # use the start of *this* interval, not the next
        t = self.rolloverAt - self.interval
        if self.utc:
            time_tuple = time.gmtime(t)
        else:
            time_tuple = time.localtime(t)
            dst = time.localtime(self.rolloverAt)[-1]
            if dst != time_tuple[-1] and self.interval > 3600:
                # DST switches between t and self.rolloverAt, adjust
                addend = 3600 if dst else -3600
                time_tuple = time.localtime(t + addend)
        return time.strftime(self.pattern, time_tuple)

    @baseFilename.setter
    def baseFilename(self, _):
        # assigned to by FileHandler, just ignore this as we use self.pattern
        # instead
        pass

    def _matching_files(self):
        """Generate DirEntry entries that match the filename pattern.

        The files are ordered by their last modification time, most recent
        files first.

        """
        matches = []
        pattern = self.pattern
        for entry in os.scandir(os.path.dirname(pattern)):
            if not entry.is_file():
                continue
            try:
                time.strptime(entry.path, pattern)
                matches.append(entry)
            except ValueError:
                continue
        matches.sort(key=lambda e: e.stat().st_mtime, reverse=True)
        return iter(matches)

    def doRollover(self):
        """Do a roll-over. This basically needs to open a new generated filename.
        """
        if self.stream:
            self.stream.close()
            self.stream = None

        if self.backupCount > 0:
            delete = islice(self._matching_files(), self.backupCount, None)
            for entry in delete:
                os.remove(entry.path)

        now = int(time.time())
        rollover = self.computeRollover(now)
        while rollover <= now:
            rollover += self.interval
        if not self.utc:
            # If DST changes and midnight or weekly rollover, adjust for this.
            if self.when == "MIDNIGHT" or self.when.startswith("W"):
                dst = time.localtime(now)[-1]
                if dst != time.localtime(rollover)[-1]:
                    rollover += 3600 if dst else -3600
        self.rolloverAt = rollover

        if not self.delay:
            self.stream = self._open()

    # borrow *some* TimedRotatingFileHandler methods
    computeRollover = TimedRotatingFileHandler.computeRollover
    shouldRollover = TimedRotatingFileHandler.shouldRollover

将其与日志文件名中的time.strftime() placeholders一起使用,这些将为您填充:

handler = TimedPatternFileHandler("logs/%H-%M.log", when="M", backupCount=2)

请注意,这会在创建实例时清除旧文件

答案 1 :(得分:1)

就像您在TimedRotatingFileHandler文档中看到的那样,您的日志文件名应该相同,以正确获取旋转系统。

在您的情况下,由于您自己要附加dateTime信息,因此日志文件名每次都不同,因此您可以观察结果。

因此,在您的源代码中,您只需要修改日志文件名:

handler = logging.handlers.TimedRotatingFileHandler(
    "logs/MyLog", 
    when="M", 
    backupCount=2)

如果要挑战它,可以将when更改为“ S”(秒),并检查旋转是否正常。

例如,它将自动生成此类文件:

> MyLog
> MyLog.2019-07-08_11-36-53
> MyLog.2019-07-08_11-36-58

如果您需要其他信息,请不要犹豫。

答案 2 :(得分:-1)

正如其他人已经指出的那样, <div id=b></div> <div id=b></div> <div id=b></div>仅在您始终登录到具有相同文件名的文件然后不时旋转时才起作用。然后,将显示类似@Bsquare的日志文件。 但是,就我而言,我需要每天轮换工作,并让我的日志文件具有以下名称:backupCount2019-07-06.log2019-07-07.log,... 我发现使用TimedRotatingFileHandler

的当前实现是不可能的

因此,我最终在FileHandler之上创建了自己的删除功能以满足我的需求 这是使用2019-07-07.log的记录器类的简单示例,可确保每次创建此类的实例时都删除旧的日志文件:

FileHandler

然后您将像这样使用它:

import os
import datetime
import logging
import re
import pathlib


class Logger:

    # Maximum number of logs to store
    LOGS_COUNT = 3

    # Directory to log to
    LOGS_DIRECTORY = "logs"

    def __init__(self):
        # Make sure logs directory is created
        self.__create_directory(Logger.LOGS_DIRECTORY)
        # Clean old logs every time you create a logger
        self.__clean_old_logs()
        self.logger = logging.getLogger("Logger")
        # If condition will make sure logger handlers will be initialize only once when this object is created
        if not self.logger.handlers:
            self.logger.setLevel(logging.INFO)
            formatter = logging.Formatter("%(asctime)s - %(levelname)s - %(message)s")
            file_handler = logging.FileHandler("logs/{:%Y-%m-%d}.log".format(datetime.datetime.now()))
            file_handler.setFormatter(formatter)
            self.logger.addHandler(file_handler)

    def log_info(self, message):
        self.logger.info(message)

    def log_error(self, message):
        self.logger.error(message)

    def __clean_old_logs(self):
        for name in self.__get_old_logs():
            path = os.path.join(Logger.LOGS_DIRECTORY, name)
            self.__delete_file(path)

    def __get_old_logs(self):
        logs = [name for name in self.__get_file_names(Logger.LOGS_DIRECTORY)
            if re.match("([12]\d{3}-(0[1-9]|1[0-2])-(0[1-9]|[12]\d|3[01]))\.log", name)]
        logs.sort(reverse=True)
        return logs[Logger.LOGS_COUNT:]

    def __get_file_names(self, path):
        return [item.name for item in pathlib.Path(path).glob("*") if item.is_file()]

    def __delete_file(self, path):
        os.remove(path)

    def __create_directory(self, directory):
        if not os.path.exists(directory):
            os.makedirs(directory)