Question

除标准输出外，我还要创建一个日志文件。我正在使用内置模块，如：

def dedup_col_col2(final_recos):
    """
    Removing duplicate combination of col1-col2 for any single Muid
    """
    recommendation_log()
    def f(data,col1, col2):
        ra=data[~(data[[col1,col2]].duplicated())]
        return ra
    .....Do something(carry on)

def function_logger(file_level, console_level = None):
    """
    Logging function to log all the warning/error messages
    It writes both to console and log file
    """
    function_name = inspect.stack()[1][3]
    logger = logging.getLogger(function_name)
    logger.setLevel(logging.DEBUG) #By default, logs all messages
    if console_level != None:
        ch = logging.StreamHandler() #StreamHandler logs to console
        ch.setLevel(console_level)
        ch_format = logging.Formatter('%(asctime)s - %(message)s')
        ch.setFormatter(ch_format)
        logger.addHandler(ch)
    fh = logging.FileHandler("{0}.log".format(function_name))
    fh.setLevel(file_level)
    fh_format = logging.Formatter('%(asctime)s - %(lineno)d - %(levelname)-8s - %(message)s')
    fh.setFormatter(fh_format)
    logger.addHandler(fh)
    return logger

def recommendation_log():
    recommendation_log_logger = function_logger(logging.DEBUG, logging.DEBUG)
    recommendation_log_logger.debug('debug message')
    recommendation_log_logger.info('info message')
    recommendation_log_logger.warn('warn message')
    recommendation_log_logger.error('error message')
    recommendation_log_logger.critical('critical message')    
def main():
    recommendation_log()  
    final_recos1=dedup_tbid_tbidrank(final_recos)
    logging.shutdown()

代码取自以下链接： How do I write log messages to a log file and the console at the same time?

现在，当我执行代码时，它会在控制台上显示各种调试错误消息。当我打开日志文件时，它会在recommendation_log.log文件中显示以下输出：

2015-07-07 12:19:16,392 - 339 - DEBUG    - debug message
2015-07-07 12:19:16,392 - 340 - INFO     - info message
2015-07-07 12:19:16,393 - 341 - WARNING  - warn message
2015-07-07 12:19:16,393 - 342 - ERROR    - error message
2015-07-07 12:19:16,393 - 343 - CRITICAL - critical message
2015-07-07 12:22:03,176 - 339 - DEBUG    - debug message
2015-07-07 12:22:03,176 - 339 - DEBUG    - debug message

现在我想了解这些错误的来源是什么，我希望看到这些消息的描述，就像它们在控制台上显示的那样。

编辑：在控制台上，它会显示警告，例如：

***Cmeans_omni.py:37: SettingWithCopyWarning: 
A value is trying to be set on a copy of a slice from a DataFrame.
Try using .loc[row_indexer,col_indexer] = value instead
See the the caveats in the documentation: http://pandas.pydata.org/pandas-docs/stable/indexing.html#indexing-view-versus-copy
  df['CampaignName_upd'] = df['CampaignName']
/home/ubuntu/anaconda/lib/python2.7/site-packages/pandas/io/parsers.py:1170: DtypeWarning: Columns (3,7) have mixed types. Specify dtype option on import or set low_memory=False.
  data = self._reader.read(nrows)***

现在我想了解这些错误的来源是什么，我也希望在日志文件中看到这些消息的描述，方式与控制台上显示的相同

基本上我想看到描述，如上面在日志文件和实际行号中所述，来自它的起源。示例输出中显示的行号不是实际行号，而是与函数中的错误对应的行的行号。

我一天都在阅读有关日志记录模块的内容，但无法找到解决方案。

Answer 1

你混合了两件事。

让我们首先专注于伐木。你的方法似乎有效，但看起来很混乱。这是相同的想法，但开销较小：

import logging

logger = logging.getLogger(__name__)
logger.setLevel(logging.DEBUG)

fmt = logging.Formatter("%(asctime)s - "
                        "%(module)s:%(lineno)s - "
                        "%(levelname)s: %(message)s")

# Loggers need handlers for handling log messages.
# Lets's add a StreamHandler for console output...
sh = logging.StreamHandler()
sh.setFormatter(fmt)
sh.setLevel(logging.DEBUG)
logger.addHandler(sh)

# ...and a FileHandler for Logfile writing:
fh = logging.FileHandler('log.file', 'a')
fh.setFormatter(fmt)
fh.setLevel(logging.WARN)
logger.addHandler(fh)

# Now let's log some stuff:
logger.debug("Foobar")
logger.info("Meh")
logger.warn("I warned you.")
logger.error("This might be an error.")

如果您遇到错误并希望在日志中包含回溯，请使用logger.exception()：

try:
    x = 1 / 0
except ZeroDivisionError as e:
    # this includes the traceback as level ERROR
    logger.exception(e)

但是，您的Error只是Warning，不会强制程序终止。如果您需要做一些与警告相关的事情，您可能需要查看warnings模块。

在这种情况下，pandas会抛出SettingWithCopyWarning，您正在为DataFrame的一部分的副本分配值。这意味着，您修改的所有内容都不会影响原始数据，因为您只使用副本。使用df.loc[] as explained in the documentation来解决此问题。

更新

使用warnings.warn()消息，您可以使用catch_warnings() - contextmanager将其添加到记录器：

import warnings

class WarningsAdapter(logging.LoggerAdapter):
    """Copy `warnings.WarningMessage` information to log record"""
    def process(self, msg, kwargs):
        if isinstance(msg, warnings.WarningMessage):
            msg = "%s: %s" % (msg.category.__name__, msg.message)
        return msg, kwargs

adapter = WarningsAdapter(logger, extra=None)

with warnings.catch_warnings(record=True) as warning_list:
    # do something throwing a warning
    warnings.warn("This is a warning message.", RuntimeWarning)
    warnings.warn("Something might have gone wrong.", UserWarning)
    # something else
    for w in warning_list:
        adapter.warning(w)

您可能需要查看this question，但warnings.warn()和logging.warning()的用例正在讨论中。

需要在python中查看各种模块的日志记录输出的描述

1 个答案:

更新