Question

我想通过EFK建立一个日志管理系统。我现在尝试了几天，以使我的多行mycat日志解析器可以使用流利的位。

我的Fluentbit配置：

configFile.load(this.getClass().getClassLoader().getResourceAsStream("/config/property_table.config.txt"));

和mycat.log看起来像这样：

parsers.conf:
    [PARSER]
      Name        mycat_error_log_parser_head
      Format      regex
      Regex       ^(?<time>(\d)+(-\d+)+(\S)+\W(\S)+)(\s+)(?<action>\S+)(\s+)(?<on>\S+)
      Time_Key    time
      Time_Format %Y-%m-%d %H:%M:%S.%L
      Time_Keep   On

    [PARSER]
      Name        mycat_error_log_parser
      Format      regex
      Regex       ^(?<time>(\d)+(-\d+)+(\S)+\W(\S)+)(\s+)(?<action>\S+)(\s+)(?<on>\S+)(\s+)(?<content>(.|\s)*)$
      Time_Key    time
      Time_Format %Y-%m-%d %H:%M:%S.%L
      Time_Keep   On

fluent-bit.conf
    [INPUT]
        Name tail
        tag mycat
        path /var/log/mycat.log
        Multiline On
        Parser_Firstline mycat_error_log_parser_head
        Parser mycat_error_log_parser
        Path_Key file

我想我现在就快要走了，但是到目前为止还没有运气。

在我的木片上。我得到以下结果： enter image description here

我的多行错误日志丢失。但是我在ruular中测试它正常解析 enter image description here

任何帮助将不胜感激。

Answer 1

此配置适用于您的日志文件，Fluent Bit发布了5个文档以使用此配置进行弹性搜索。

Firefox hook launcher starting up
run-parts: executing /usr/lib/ppt/firefox-hooks/00ppt-firefox-certs
Notice: Trust flag u is set automatically if the private key is present.
Notice: Trust flag u is set automatically if the private key is present.
Notice: Trust flag u is set automatically if the private key is present.
Notice: Trust flag u is set automatically if the private key is present.
Notice: Trust flag u is set automatically if the private key is present.
Notice: Trust flag u is set automatically if the private key is present.
Notice: Trust flag u is set automatically if the private key is present.
Notice: Trust flag u is set automatically if the private key is present.
Notice: Trust flag u is set automatically if the private key is present.
Notice: Trust flag u is set automatically if the private key is present.
Notice: Trust flag u is set automatically if the private key is present.
Notice: Trust flag u is set automatically if the private key is present.
Notice: Trust flag u is set automatically if the private key is present.
Notice: Trust flag u is set automatically if the private key is present.
Notice: Trust flag u is set automatically if the private key is present.
run-parts: executing /usr/lib/ppt/firefox-hooks/10firefox-disable-bad-prefs
Firefox hook launcher finished

问题中对配置进行了两项更改-[PARSER]部分中的Regex配置已更改，[INPUT]部分中的Parser更改为Parser_1。

我在多行解析中也遇到了类似的问题，我的观察是Parser_Firstline应该覆盖多行消息中的整个第一行，而Parser_1 .. Parser_N应该覆盖多行消息的整个结构。

有关多行配置的Fluent Bit文档：https://fluentbit.io/documentation/0.12/input/tail.html

Answer 2

请从Parser更改为Parser_1。

Parser_Firstline必须是匹配除多行例外之外的所有内容的正则表达式和Parser_1必须是匹配所有（包括多行例外）的正则表达式。

带有Mycat多行解析的Fluentbit

2 个答案: