无法在Logstash

Question

我已经嵌套了json来解析logstash。我的日志看起来像：

{
    "schema": {
        "type": "struct",
        "fields": [{
            "type": "string",
            "optional": false,
            "field": "event"
        }, {
            "type": "int16",
            "optional": false,
            "field": "partition"
        }, {
            "type": "string",
            "optional": false,
            "field": "key"
        }, {
            "type": "int64",
            "optional": false,
            "field": "cas"
        }, {
            "type": "int64",
            "optional": false,
            "field": "bySeqno"
        }, {
            "type": "int64",
            "optional": false,
            "field": "revSeqno"
        }, {
            "type": "int32",
            "optional": true,
            "field": "expiration"
        }, {
            "type": "int32",
            "optional": true,
            "field": "flags"
        }, {
            "type": "int32",
            "optional": true,
            "field": "lockTime"
        }, {
            "type": "string",
            "optional": true,
            "field": "content"
        }],
        "optional": false,
        "name": "com.couchbase.DcpMessage"
    },
    "payload": {
        "event": "mutation",
        "partition": 272,
        "key": "Logs::1295f07e-09c0-4a4c-bb2e-fc5240fb5232",
        "cas": 1503898266643070976,
        "bySeqno": 1,
        "revSeqno": 1,
        "expiration": 0,
        "flags": 0,
        "lockTime": 0,
        "content": "{\"_sync\":{\"rev\":\"1-ca7cfba46d5e2567a41092e579940748\",\"sequence\":698,\"recent_sequences\":[698],\"history\":{\"revs\":[\"1-ca7cfba46d5e2567a41092e579940748\"],\"parents\":[-1],\"channels\":[null]},\"time_saved\":\"2017-08-28T05:30:43.587693145Z\"},\"componentName\":\"Virgin Voyages\",\"componentVersion\":\"1.2.5\",\"corelationId\":\"1503897743127\",\"deviceinfo\":{\"id\":\"5c9142899222677d\",\"name\":\"samsung SM-G900M\",\"operatingSystem\":\"Android\",\"osVersion\":\"6.0.1\",\"type\":\"Phone\"},\"hostname\":\"samsung SM-G900M\",\"message\":null,\"messageCode\":\"INJECT\",\"messageDetail\":\"{\\\"informationType\\\":\\\"beaconDetails\\\",\\\"actionData\\\":[{\\\"beacon_id\\\":\\\"b9407f30-f5f8-466e-aff9-25556b57fe6d:19755:58997\\\",\\\"distance\\\":3.346020288426232},{\\\"beacon_id\\\":\\\"b9407f30-f5f8-466e-aff9-25556b57fe6d:20063:45814\\\",\\\"distance\\\":8.37788453900756},{\\\"beacon_id\\\":\\\"b9407f30-f5f8-466e-aff9-25556b57fe6d:24416:27026\\\",\\\"distance\\\":8.170533849237675},{\\\"beacon_id\\\":\\\"b9407f30-f5f8-466e-aff9-25556b57fe6d:38529:59955\\\",\\\"distance\\\":4.72251958192618},{\\\"beacon_id\\\":\\\"b9407f30-f5f8-466e-aff9-25556b57fe6d:40536:39585\\\",\\\"distance\\\":4.699073594284207},{\\\"beacon_id\\\":\\\"b9407f30-f5f8-466e-aff9-25556b57fe6d:40795:1374\\\",\\\"distance\\\":9.36269856965088},{\\\"beacon_id\\\":\\\"b9407f30-f5f8-466e-aff9-25556b57fe6d:42423:37145\\\",\\\"distance\\\":2.5632779653262023},{\\\"beacon_id\\\":\\\"b9407f30-f5f8-466e-aff9-25556b57fe6d:44572:21424\\\",\\\"distance\\\":5.1373696851890704},{\\\"beacon_id\\\":\\\"b9407f30-f5f8-466e-aff9-25556b57fe6d:51771:64898\\\",\\\"distance\\\":3.9813075660947947},{\\\"beacon_id\\\":\\\"b9407f30-f5f8-466e-aff9-25556b57fe6d:52163:7961\\\",\\\"distance\\\":6.976285634989775},{\\\"beacon_id\\\":\\\"b9407f30-f5f8-466e-aff9-25556b57fe6d:5569:23454\\\",\\\"distance\\\":5.350671148905375},{\\\"beacon_id\\\":\\\"b9407f30-f5f8-466e-aff9-25556b57fe6d:60383:48481\\\",\\\"distance\\\":5.1373696851890704},{\\\"beacon_id\\\":\\\"b9407f30-f5f8-466e-aff9-25556b57fe6d:6949:14480\\\",\\\"distance\\\":4.337792827947945},{\\\"beacon_id\\\":\\\"f7826da6-4fa2-4e98-8024-bc5b71e0893e:29949:42958\\\",\\\"distance\\\":3.6595317170226993},{\\\"beacon_id\\\":\\\"f7826da6-4fa2-4e98-8024-bc5b71e0893e:32521:11968\\\",\\\"distance\\\":5.0375909818338265}]}\",\"offset\":\"+05:30\",\"severity\":\"Debug\",\"stacktrace\":null,\"timestamp\":\"2017-08-28T10:52:23.126\",\"traceinfo\":{\"duration\":\"0\",\"loggingPoint\":null,\"methodTime\":\"2017-08-28T10:52:23.126\"},\"type\":\"log\",\"userinfo\":{\"appId\":null,\"deviceId\":null,\"id\":null,\"token\":null,\"type\":null}}"
    }
}

当我将json滤镜应用为source =＆gt; “消息”，我的日志被解析但在我的payload.content字段中，还有另一个json，我也想解析它。我在source =＆gt;上尝试了json filter “[消息] [有效载荷] [内容]”但之后没有成功。它无法解析我的payload.content字段的嵌套json。请指导我。

Answer 1

如果你想解析嵌套的json（在你的情况下是payload.content），你可以使用过滤器序列，如下所示：

filter {
    json {
        source => "message"
        target => "parsedMain"
    }
    json {
        source => "[parsedMain][payload][content]"
        target => "parsedContent"
    }
}

更新：我尝试了更简单的测试：

{"test":"1234", "payload":{"content":"{\"innerTest\":\"12345\"}"}}

结果：

{
   "@timestamp" => 2017-09-01T07:23:01.402Z,
"parsedContent" => {
    "innerTest" => "12345"
},
     "@version" => "1",
         "host" => "...",
   "parsedMain" => {
       "test" => "1234",
    "payload" => {
        "content" => "{\"innerTest\":\"12345\"}"
    }
},
      "message" => "{\"test\":\"1234\", \"payload\":{\"content\":\"{\\\"innerTest\\\":\\\"12345\\\"}\"}}"

}