Question

以下是我的logstash配置，用于加载弹性搜索数据并转换为csv格式

input {
elasticsearch {
hosts => "localhost:9200"
index => "chats"
query => '{ "query": { "range" : {
"timestamp" : {
"gte" : "1492080665000",
"lt" : "1492088665000"
}
} }, "_source": [ "timestamp","content.text"] }'
}
}

filter {
date {
match => [ "timestamp","UNIX_MS" ]
target => "timestamp_new"
remove_field => [ "timestamp" ]
}
csv {
columns => ["timestamp", "content.text"]
separator => ","
}
}

output{
csv {
fields => ["timestamp_new","content.text"]
path => "/home/ubuntu/chats-content-date-range-v3.csv"
}
stdout { codec => rubydebug }
}

示例输入数据

"source":{"userName": "xxx", "senderType": 3, "spam": 0, "senderId": "1000", "threadId": 101, "userId": "xxx", "sessionId": 115, "content": {"text": "Yes okay", "image": null, "location": null, "card": null}, "receiverId": "xxx", "timestamp": 1453353242657, "type": 0, "id": "0dce30dd-781e-4a42-b230-a988b68fd9ed1000_1453353242657"}

以下是我的示例输出数据

2017-04-13T12:41:34.423Z,"{""text"":""Yes okay""}"

相反，我想要关注输出

2017-04-13T12:41:34.423Z,"Yes okay"

Answer 1

input {
    elasticsearch {
        hosts => "localhost:9200"
        index => "chats"
        query => '{
            "query": { 
                "range" : {
                    "timestamp" : {
                        "gte" : "1492080665000",
                        "lt" : "1492088665000"
                    }
                } 
            }, 
            "_source": [ "timestamp","content.text"] 
        }'
    }
}

filter {
    date {
        match => [ "timestamp","UNIX_MS" ]
        target => "timestamp_new"
        remove_field => [ "timestamp" ]
    }
    csv {
        columns => ["timestamp", "content.text"]
        separator => ","
    }
    json {
        source => "content.text"
        target => "content.text"
    }
}

Logstash csv配置访问嵌套的json字段

1 个答案: