Question

嗨，我有来自springboot微服务的以下日志。在弹性以下所有字段（如时间戳，级别，记录器等）上创建索引的内容。如何在流利的配置中实现这一目标？尝试了以下内容，但没有成功

登录

timestamp:2020-04-27 09:37:56.996 level:INFO level_value:20000 thread:http-nio-8080-exec-2 logger:com.scb.nexus.service.phoenix.components.ApplicationEventListener context:default message:org.springframework.web.context.support.ServletRequestHandledEvent traceId:a122e51aa3d24d4a spanId:a122e51aa3d24d4a spanExportable:false X-Span-Export:false X-B3-SpanId:a122e51aa3d24d4a X-B3-TraceId:a122e51aa3d24d4a

流利的会议

 <match **>
      @type elasticsearch
      time_as_integer true
      include_timestamp true
      host host
      port 9200
      user userName
      password password
      scheme https
      ssl_verify false
      ssl_version TLSv1_2
      index_name testIndex
    </match>
    <filter **>
      @type parser
      key_name log
      reserve_data true
      <parse>
        @type json
      </parse>
    </filter>

Answer 1

日志不是JSON格式，因此您不能使用Json解析器。您可以使用以下选项来解决此问题

1-使用正则表达式解析器，如此处https://docs.fluentd.org/parser/regexp所述 2-使用record_reformer插件并手动提取项目

示例：

 <match **>
    @type record_reformer
    tag parsed.${tag_suffix[2]}
    renew_record false
    enable_ruby true
    <record>
       timestamp ${record['log'].scan(/timestamp:(?<param>[^ ]+ [^ ]+)/).flatten.compact.sort.first}
       log_level ${record['log'].scan(/level:(?<param>[^ ]+)/).flatten.compact.sort.first}
       level_value ${record['log'].scan(/level_value:(?<param>[^ ]+)/).flatten.compact.sort.first}
   </record>
  </match>

 <match parsed.**>
  @type elasticsearch
  time_as_integer true
  include_timestamp true
  host host
  port 9200
  user userName
  password password
  scheme https
  ssl_verify false
  ssl_version TLSv1_2
  index_name testIndex
</match>

FluentD配置可在弹性日志中索引所有字段

1 个答案: