Question

我的情况是，我有一个配置文件，显然，每次系统运行时都会导入/调用。

在我的配置文件中，我有一个带有日志记录配置的dict，在程序开头我得到了这个配置，并用dict执行logging.config.dictConfig（）。

我的系统在子进程中运行，至少有100名工作人员使用RQ。有时，当系统运行时，我不断收到错误：

Traceback (most recent call last):
File "/home/manutencao/Heimdall/heimdall/worker.py", line 146, in heimdall_tradutor
File "/usr/local/lib/python3.5/logging/__init__.py", line 1279, in info
File "/usr/local/lib/python3.5/logging/__init__.py", line 1415, in _log
File "/usr/local/lib/python3.5/logging/__init__.py", line 1425, in handle
File "/usr/local/lib/python3.5/logging/__init__.py", line 1487, in callHandlers
File "/usr/local/lib/python3.5/logging/__init__.py", line 855, in handle
File "/home/manutencao/Heimdall/heimdall/logger.py", line 28, in emit
File "/home/manutencao/Ratatosk/ratatosk/__init__.py", line 69, in __init__
File "/home/manutencao/Ratatosk/ratatosk/__init__.py", line 133, in get_logger
File "/usr/local/lib/python3.5/logging/handlers.py", line 150, in __init__
File "/usr/local/lib/python3.5/logging/handlers.py", line 57, in __init__
File "/usr/local/lib/python3.5/logging/__init__.py", line 1008, in __init__
File "/usr/local/lib/python3.5/logging/__init__.py", line 1037, in _open
OSError: [Errno 24] Too many open files: '/home/manutencao/Ratatosk/ratatosk/JobRQ.log'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/home/manutencao/.virtualenvs/heimdall/lib/python3.5/site-packages/rq/worker.py", line 588, in perform_job
File "/home/manutencao/.virtualenvs/heimdall/lib/python3.5/site-packages/rq/job.py", line 498, in perform
File "/home/manutencao/Heimdall/heimdall/worker.py", line 195, in heimdall_tradutor
File "/usr/local/lib/python3.5/logging/__init__.py", line 1314, in exception
File "/usr/local/lib/python3.5/logging/__init__.py", line 1308, in error
File "/usr/local/lib/python3.5/logging/__init__.py", line 1415, in _log
File "/usr/local/lib/python3.5/logging/__init__.py", line 1425, in handle
File "/usr/local/lib/python3.5/logging/__init__.py", line 1487, in callHandlers
File "/usr/local/lib/python3.5/logging/__init__.py", line 855, in handle
File "/home/manutencao/Heimdall/heimdall/logger.py", line 28, in emit
File "/home/manutencao/Ratatosk/ratatosk/__init__.py", line 69, in __init__
File "/home/manutencao/Ratatosk/ratatosk/__init__.py", line 133, in get_logger
File "/usr/local/lib/python3.5/logging/handlers.py", line 150, in __init__
File "/usr/local/lib/python3.5/logging/handlers.py", line 57, in __init__
File "/usr/local/lib/python3.5/logging/__init__.py", line 1008, in __init__
File "/usr/local/lib/python3.5/logging/__init__.py", line 1037, in _open
OSError: [Errno 24] Too many open files: '/home/manutencao/Ratatosk/ratatosk/JobRQ.log'

我有一个自定义记录器处理程序：

class RQHandler(logging.Handler):  # Inherit from logging.Handler
    def __init__(
        self, formatter=JSONFormatter(), level=logging.NOTSET,
        connection_pool=None
    ):
        # run the regular Handler __init__
        logging.Handler.__init__(self, level)
        self.formatter = formatter
        self.connection_pool = connection_pool

    def emit(self, record):
        # record.message is the log message
        ratatosk = Ratatosk(
            all_queues={'huginn_log': [1, 1]},
            debug=configuracoes['level_log'],
            REDIS_HOST=conf_redis['host'],
            REDIS_PORT=conf_redis['port'],
            REDIS_DB=conf_redis['db'],
            connection_pool=(
                self.connection_pool or conf_redis.get('connection_pool')
            )
        )
        ratatosk.enqueue(
            'huginn_log',
            'huginn.service.register_log',
            (self.format(record)),
        )

基本上，我只需获取记录器并将其添加到RQ队列。

我已经读过，当您在日志记录中有太多处理程序时会导致此错误。这是因为我在我的代码的开头添加了logging.config.dictConfig（）吗？

提前致谢

Answer 1

这很可能意味着在RataTosk的实现中有一个与Redis或文件的未关闭连接的错误（套接字计为linux中的一个打开文件）。如果您对实现有信心，可以在linux中增加文件限制：

ulimit -Hn
ulimit -Sn
# increase the fs.file-max in /etc/sysctl.conf to *2 of greater of the two
# restart the process hosting your python app

Python日志记录获取错误：打开的文件太多

1 个答案: