Question

我想将json文档从一种结构转换为另一种结构。我已经尝试通过多种解决方法来解决它，但是不知何故我没有得到预期的输出。

我有一个json日志文件，如下所示：

 {
    "consumer": {
        "created_at": 1566912618154,
        "username": "dummyapp",
        "id": "07912445-ca35-464b-8596-b2ace5e60481"
    },
    "service": {
        "created_at": 1567173649,
        "connect_timeout": 60000,
        "protocol": "https",
        "read_timeout": 60000,
        "port": 9090,
        "updated_at": 1567173649,
        "retries": 1,
        "write_timeout": 60000
    },
    "request": {
        "querystring": {},
        "size": "361",
        "headers": {
            "cache-control": "no-cache",
            "content-length": "0",
            "postman-token": "fb9e10e4-2f66-4126-beec-d5c1f7c06bf7",
            "user-agent": "PostmanRuntime/7.15.0",
            "accept": "/",
            "accept-encoding": "gzip, deflate",
            "connection": "keep-alive"
        },
        "method": "POST"
    }
}

我想将其转换为如下形式：

{
    "Title":"Sample"
    "consumer": {
        "created_at": 1566912618154,
        "username": "dummyapp"
    },
    "service": {
        "created_at": 1567173649,
        "connect_timeout": 60000
    },
    "request": {
        "querystring": {},
        "headers": {
            "user-agent": "PostmanRuntime\/7.15.0"
        }
        "method": "POST"
    }
}

这是我的管道配置：

input {
    file {
        path => "Path_to_Log\test.log"
        start_position => "beginning"
        type => "json"
        codec => "json"
    }
}

filter {
    mutate {
        add_field => {
            "Title" => "Sample"
        }
        add_field => {
            "consumer" => "%{[consumer]}"
        }
        ...further changes
    }
}

output {
    file {
        path => "Output_Path\Output.log"
    }
}

这是我得到的答复：

    { 
   "@timestamp":"2019-09-10T09:20:38.569Z",
   "Title":"Sample",
   "@version":"1",
   "consumer":[ 
      [ 
         "created_at",
         1566912618154
      ],
      [ 
         "username",
         "dummyapp"
      ],
      "{\"created_at\":1566912618154,\"username\":\"dummyappapp\"}"
   ],
   "type":"json"
   ..some data
}

为什么我会得到上面的输出？
如何转换json文档来满足上述要求？
如何设置从源到目标子元素的标签？

Answer 1

那么为什么不删除不感兴趣的字段呢？

filter {
    mutate {
        add_field => {
            "Title" => "Sample"
        }

        remove_field => [
          "[consumer][id]",
          "[service][protocol]",
          # and so on for the service element
          "[request][size]",
          "[request][headers][cache-control]",
          # and so on for the request.headers element
        ]

        # THIS IS OBSOLETE NOW!
        #add_field => {
        #    "consumer" => "%{[consumer]}"
        #}
        ...further changes
    }
}

有关如何访问事件数据和字段（尤其是嵌套的字段）的更多信息，请参考this guide。

Logstash中的Json转换

1 个答案: