我的JSON输入如下,其中包含日期字段,需要从Json中提取日期时间字段,
{
"Properties": {
"Client Name": "Chubb",
"Portfolio": "Chubb-Transfer"
},
"Capture": [
{
"CaptureGUID": "caa1f5ba-1e93-4926-b3ac-e30d0d9d4cbb",
"HTMLPath": "Captures\\C:\\",
"ScreenName": "Amdocs CRM - ClearCallCenter - [Console]",
"TimeStamp": "20170926110036"
},
{
"CaptureGUID": "0faf6b54-999f-4bfd-b8d0-e81a589f9185",
"HTMLPath": "Captures\\C:\\",
"ScreenName": "Microsoft Excel - 1.0.1 1.0.6 1.0.8 Match 3.0.6 Hit NAIC Optimized.xlsx",
"TimeStamp": "20170926105418"
}
]
}
和我的Logstash配置如下,如何将字符串日期(" TimeStamp":" 20170926105418")转换为日期格式。已更新完整的Logstash文件
input {
file {
type => "json"
path => "C:/ELK/data/Recordings/*.json"
start_position => beginning
codec => multiline {
pattern => "^{"
negate => "true"
what => "previous"
multiline_tag => "multi_tagged"
max_lines => 30000
}
}
}
filter{
date {
match => ["Capture.TimeStamp", "yyyyMMddHHmmss"]
target => "TimeStamp"
}
mutate {
replace => { "message" => "%{message}}" }
gsub => [ 'message','\n','']
}
json {
source => "message"
remove_field => ["message"]
}
}
output {
elasticsearch {
index => "test10"
}
stdout { codec => rubydebug }
}
答案 0 :(得分:1)
从logstash配置文件中删除日期过滤器。映射索引时处理日期解析。以下是您的用例的映射。
PUT json
{
"mappings": {
"json": {
"properties": {
"Capture": {
"type": "nested",
"properties": {
"CaptureGUID": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"HTMLPath": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"ScreenName": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"TimeStamp": {
"type": "date",
"format": "yyyyMMddHHmmss"
}
}
},
"Properties": {
"properties": {
"Client Name": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"Portfolio": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
}
}
}
}
}
}
}
答案 1 :(得分:0)
已通过以下方式解决,
input {
file {
type => "json"
path => "C:/ELK/data/Recordings/*.json"
start_position => beginning
codec => multiline {
pattern => "^{"
negate => "true"
what => "previous"
max_lines => 30000
}
}
}
filter{
mutate {
replace => { "message" => "%{message}}" }
gsub => [ 'message','\n','']
}
json {
source => "message"
remove_field => ["message"]
}
date {
match => ["[Capture][0][TimeStamp]", "yyyyMMddHHmmss"]
target=> "[Capture][0]StartTime"
timezone => "Africa/Lome"
locale => "en"
}
}
output {
elasticsearch {
index => "test15"
}
stdout { codec => rubydebug }
}