Question

我正在使用Logstash将JSON消息输出到API。我正在使用“mapping”属性来映射我的消息。请参阅我的托运人配置。

output {
    stdout { }
     http {
        url => "http://localhost:8087/messages"
        http_method => "post"
        format => "json"
        mapping => ["MessageId","654656","TimeStamp","2001-12-31T12:00:00","CorrelationId","986565","MessageType","%{log_MessageType}" ,"MessageTitle","%{log_MessageTitle}","Message","%{log_Message}"]
    }
}

此配置工作正常，并产生以下输出：

{
  "MessageId": "654656",
  "TimeStamp": "2001-12-31T12:00:00",
  "CorrelationId": "986565",
  "MessageType": "INFO",
  "MessageTitle": "TestTittle",
  "Message": "Sample Message"
}

输入日志条目：

TID: [0] [ESB] [2016-05-30 23:02:02,602]  INFO {org.wso2.carbon.registry.core.jdbc.EmbeddedRegistryService} -  Configured Registry in 572ms {org.wso2.carbon.registry.core.jdbc.EmbeddedRegistryService}

Grok模式：

TID:%{SPACE}\[%{INT:log_SourceSystemId}\]%{SPACE}\[%{DATA:log_ProcessName}\]%{SPACE}\[%{TIMESTAMP_ISO8601:log_TimeStamp}\]%{SPACE}%{LOGLEVEL:log_MessageType}%{SPACE}{%{JAVACLASS:log_MessageTitle}}%{SPACE}-%{SPACE}%{GREEDYDATA:log_Message}

问题陈述：

我希望通过HTTP映射关注输出。我想在我的消息中使用嵌套的JSON类型，我应该如何在映射标记中添加它。

预期输出：

{
  "MessageId": "654656",
  "TimeStamp": "2001-12-31T12:00:00",
  "CorrelationId": "986565",
  "MessageType": "INFO",
  "MessageTitle": "TestTittle",
  "Message": "Sample Message",
  "MessageDetail": {
    "FieldA": "65656",
    "FieldB": "192.168.1.1",
    "FieldC": "sample value"

  }
}

我尝试过几个选项，但收到错误。

Answer 1

使用message输出中的http映射无法执行此操作。该映射只能创建单级JSON。

但是，您可以使用http过滤器在JSON消息到达mutate/add_field输出之前构建它。

filter {
   grok {
       match => { "message" => "TID:%{SPACE}\[%{INT:SourceSystemId}\]%{SPACE}\[%{DATA:ProcessName}\]%{SPACE}\[%{TIMESTAMP_ISO8601:log_TimeStamp}\]%{SPACE}%{LOGLEVEL:log_MessageType}%{SPACE}{%{JAVACLASS:log_MessageTitle}}%{SPACE}-%{SPACE}%{GREEDYDATA:log_Message}" }
   }

   # add additional fields in your event here
   mutate {
      gsub => [
        "log_TimeStamp", "\s", "T",
        "log_TimeStamp", ",", "."
      ]
      add_field => {
        "MessageId" => "654656"
        "TimeStamp" => "%{log_TimeStamp}"
        "CorrelationId" => "986565"
        "MessageType" => "%{log_MessageType}"
        "MessageTitle" => "%{log_MessageTitle}"
        "Message" => "%{log_Message}"
        "[MessageDetail][FieldA]" => "65656"
        "[MessageDetail][FieldB]" => "192.168.1.1"
        "[MessageDetail][FieldC]" => "sample value"
      }
      remove_field => ["@version", "@timestamp", "host", "message", "SourceSystemId", "ProcessName", "log_TimeStamp", "log_MessageType", "log_MessageTitle", "log_Message"]
   }
}
output {
   stdout { codec => "rubydebug" }
   http {
      url => "http://localhost:8087/messages"
      http_method => "post"
      format => "json"
   }
}

您将获得您希望发布到HTTP端点的JSON

{
         "MessageId": "654656",
         "TimeStamp": "2016-05-30T23:02:02.602",
     "CorrelationId": "986565",
       "MessageType": "INFO",
      "MessageTitle": "org.wso2.carbon.registry.core.jdbc.EmbeddedRegistryService",
           "Message": "Configured Registry in 572ms {org.wso2.carbon.registry.core.jdbc.EmbeddedRegistryService}",
     "MessageDetail": {
        "FieldA": "65656"
        "FieldB": "192.168.1.1"
        "FieldC": "sample value"
     }
}

如何在Log-stash HTTP输出中映射嵌套的JSON

1 个答案: