Question

题如何解析Java / Scala中的非结构化Log4J日志。背景关于如何解析Log4J日志有很多问题，但大多数人建议使用XML或JSON appender输出结构化输出来解析。但是需要以编程方式解析已生成的非结构化日志文件，而不是使用logstash，chainsaws GUI等工具。 LogFilePatternReceiver的帮助显示方式，但它需要“插件”支持，这应该在Log4J 1.3中实现，但Log4J 1.x停止使用1.2.x.它看起来Log4J 2没有LogFilePatternReceiver。＆lt;？xml version =“1.0”encoding =“UTF-8”？＆gt; ＆lt;！DOCTYPE log4j：配置＆gt; ＆lt; log4j：configuration xmlns：log4j =“http://jakarta.apache.org/log4j/”debug =“true”＆gt; ＆lt; appender name =“CONSOLE”class =“org.apache.log4j.ConsoleAppender”＆gt; ＆lt; layout class =“org.apache.log4j.SimpleLayout”/＆gt; ＆lt; filter class =“org.apache.log4j.filter.ExpressionFilter”＆gt; ＆lt; param name =“Expression”value =“msg~ ='SPECIFY CASE-INSENSITIVE MSG MATCH HERE'”/＆gt; ＆lt; param name =“AcceptOnMatch”value =“true”/＆gt; ＆LT; /滤光器＆gt; ＆lt; filter class =“org.apache.log4j.filter.DenyAllFilter”/＆gt; ＆LT; /附加器＆GT; ＆lt; plugin name =“LOGFILE”class =“org.apache.log4j.varia.LogFilePatternReceiver”＆gt; ＆lt; param name =“fileURL”value =“file：/// c：/some.log”/＆gt; ＆lt; param name =“timestampFormat”value =“yyyy-MM-dd HH：mm：ss，SSS”/＆gt; ＆lt; param name =“logFormat”value =“TIMESTAMP LEVEL [LOGGER] MESSAGE”/＆gt; ＆lt; param name =“name”value =“我的日志文件”/＆gt; ＆lt; param name =“tailing”value =“true”/＆gt; ＆LT; /插件＆GT; ＆LT;根＆GT; ＆lt; level value =“DEBUG”/＆gt; ＆lt; appender-ref ref =“CONSOLE”/＆gt; ＆LT; /根＆GT; ＆LT; / log4j的：结构＆gt; org.apache.log4j.lf5.util.LogFileParse包中有一个类LogFileParser，但不知道如何使用它。有一个工具OtrosLogViewer，但如果可能的话，不希望有第三方依赖。选项 Re：从控制台上的每个输出日志行中提取日志信息建议参考Chainsaw代码。阅读代码并实现类似的选项，但如果已经有一种已知的方法来解析日志，那么首先想知道它。参考解析log4j日志文件 LogFilePatternReceiver类

Answer 1

使用log4j和extras 1.2 jar找到了一种方法，但不能与Log4J 2一起使用。

<强>爪哇

import org.apache.log4j.Logger;
import org.apache.log4j.spi.LoggingEvent;
import org.apache.log4j.receivers.varia.LogFilePatternReceiver;

public class LogParser extends LogFilePatternReceiver {
    final static Logger logger = Logger.getLogger(LogParser.class);

    public void doPost(LoggingEvent event) {
        System.out.println(event.getTimeStamp());
        System.out.println(event.getMessage());
    }
    public static void main(String[] args) {
        (new LogParser()).run();
    }
    private void run(){
        LogFilePatternReceiver r = new LogParser();
        r.setLogFormat("TIMESTAMP LEVEL [THREAD] CLASS (FILE:LINE) - MESSAGE");
        r.setFileURL("file:///tmp/log");
        r.setTimestampFormat("yyyy-MM-dd HH:mm:ss,SSS");
        r.setTailing(true);
        r.setLoggerRepository(logger.getLoggerRepository());

        r.activateOptions();
    }
}

<强>的log4j.xml

<?xml version="1.0" encoding="UTF-8" ?>
<!DOCTYPE log4j:configuration SYSTEM "log4j.dtd">

<log4j:configuration xmlns:log4j="http://jakarta.apache.org/log4j/">
    <appender name="console" class="org.apache.log4j.ConsoleAppender">
        <param name="Target" value="System.out"/>
        <layout class="org.apache.log4j.PatternLayout">
            <param name="ConversionPattern" value="%d %-5p [%t] %C{2} (%F:%L) - %m%n"/>
        </layout>
    </appender>
    <root>
        <priority value ="debug" />
        <appender-ref ref="console" />
    </root>
</log4j:configuration>

示例日志文件

2017-12-02 16:53:42,618 DEBUG [main] LogParser (LogParser.java:15) - message 1
2017-12-02 16:53:42,624 DEBUG [main] LogParser (LogParser.java:16) - message 2

<强>结果

2017-12-02 19:21:31,633 INFO  [main] spi.Log4JULogger (Log4JULogger.java:104) - activateOptions
2017-12-02 19:21:31,652 DEBUG [LogFilePatternReceiver-plugin] spi.Log4JULogger (Log4JULogger.java:58) - regexp is (\S+-\S+-\S+ \S+:\S+:\S+,\S+)[ ]+(\S*\s*?)[ ]+\[(.*?)\][ ]+(.*?)[ ]+\((.*?):(.*?)\)[ ]+\-[ ]+(.*)
2017-12-02 19:21:31,652 INFO  [LogFilePatternReceiver-plugin] spi.Log4JULogger (Log4JULogger.java:104) - attempting to load file: file:///tmp/log

1512194022618
message 1
1512194022624
message 2

2017-12-02 19:21:33,656 DEBUG [LogFilePatternReceiver-plugin] spi.Log4JULogger (Log4JULogger.java:58) - tailing file
2017-12-02 19:21:35,660 DEBUG [LogFilePatternReceiver-plugin] spi.Log4JULogger (Log4JULogger.java:58) - tailing file

Java或Scala中的Log4j非结构化日志解析器

1 个答案: