我注意到由filebeat正确定义的@timestamp字段由logstash自动更改，其值将替换为日志时间戳值（字段名称为a_timestamp）。 以下是logstash调试日志的一部分：
[2017-07-18T11：55：03,598] [DEBUG] [logstash.pipeline]过滤器已收到{＆＃34; event＆＃34; =＆gt; {＆＃34; @ ****时间戳＆ ＃34; =＆gt; 2017-07-18T09：54：53.507Z ，＆＃34;偏移＆＃34; =＆gt; 498，＆＃34; @版本＆＃34; =＆gt;＆＃34; 1＆＃34;，＆＃34; input_type＆＃34; =＆gt;＆＃34; log＆＃34;，＆＃34; beat＆＃34; =＆gt; {＆＃34;主机名＆＃34; =＆gt;＆＃ 34; centos-ea＆＃34;，＆＃34; name＆＃34; =＆gt;＆＃34; filebeat_shipper_kp＆＃34;，＆＃34;版本＆＃34; =＆gt;＆＃34; 5.5.0＆＃34; }，＆＃34; host＆＃34; =＆gt;＆＃34; centos-ea＆＃34;，＆＃34; source＆＃34; =＆gt;＆＃34; / home / elastic / ELASTIC_NEW / log_bw / test。 log＆＃34;，＆＃34; message＆＃34; =＆gt;＆＃34; 2017-06-05 19：02：46.779 INFO [bwEngThread：In-Memory Process Worker-4] psg.logger - a_applicationName = \＆ ＃34; PieceProxy \＆＃34;，a_processName = \＆＃34; piece.PieceProxy \＆＃34;，a_jobId = \＆＃34; bw0a10ao \＆＃34;，a_processInstanceId = \＆＃34; bw0a10ao \＆ ＃34;，a_level = \＆＃34; Info \＆＃34;，a_phase = \＆＃34; ProcessStart \＆＃34;，a_activityName = \＆＃34; SetAndLog \＆＃34;， a_timeStamp = \＆＃34; 2017-06-05T19：02：46.779 \＆＃34;，a_sessionId = \＆＃34; \＆＃34; ，a_sender = \＆＃34; PCS \＆＃34;，a_cruid = \＆＃34; 37d7e225-bbe5-425b-8abc-f4b44a5a1560 \＆＃34;，a_MachineCode = \＆＃34; CFDM7757 \＆＃34; ，a_correlationId = \＆＃34; fa10f \＆＃34;，a_trackingId = \＆＃34; 9d3b8 \＆＃34;，a_message = \＆＃34; START = piece.PieceProxy \＆＃34;＆＃34; ，＆＃34;输入＆＃34; =＆gt;＆＃34; log＆＃34;，＆＃34;标签＆＃34; =＆gt; [＆＃34; beats_input_codec_plain_applied＆＃34;]}}
[2017-07-18T11：55：03,629] [DEBUG] [logstash.pipeline]输出收到{＆＃34; event＆＃34; =＆gt; {＆＃34; a_message＆＃34; =＆gt;＆＃ 34; START = piece.PieceProxy＆＃34;，＆＃34; log＆＃34; =＆gt;＆＃34; INFO＆＃34;，＆＃34; bwthread＆＃34; =＆gt;＆＃34; [bwEngThread： In-Memory Process Worker-4]＆＃34;，＆＃34; logger＆＃34; =＆gt;＆＃34; psg.logger＆＃34;，＆＃34; a_correlationId＆＃34; =＆gt;＆＃34 ; fa10f＆＃34;，＆＃34; source＆＃34; =＆gt;＆＃34; /home/elastic/ELASTIC_NEW/log_bw/test.log" ;,＆＃34; a_trackingId＆＃34; =＆gt;＆＃ 34; 9d3b8＆＃34;，＆＃34;键入＆＃34; =＆gt;＆＃34; log＆＃34;，＆＃34; a_sessionId＆＃34; =＆gt;＆＃34; \＆＃34; \＆＃ 34;＆＃34;，＆＃34; a_sender＆＃34; =＆gt;＆＃34; PCS＆＃34;，＆＃34; @ version＆＃34; =＆gt;＆＃34; 1＆＃34;，＆＃ 34; beat＆＃34; =＆gt; {＆＃34;主机名＆＃34; =＆gt;＆＃34; centos-ea＆＃34;，＆＃34;名称＆＃34; =＆gt;＆＃34; filebeat_shipper_kp＆＃34 ;，＆＃34;版本＆＃34; =＆gt;＆＃34; 5.5.0＆＃34;}，＆＃34;主机＆＃34; =＆gt;＆＃34; centos-ea＆＃34;，＆＃34 ; a_level＆＃34; =＆gt;＆＃34;信息＆＃34;，＆＃34; a_processName＆＃34; =＆gt;＆＃34; piece.PieceProxy＆＃34;，＆＃34; a_cruid＆＃34; =＆gt; ＆＃34; 37d7e225-bbe5-425b-8abc-f4b44a5a1560＆＃34;，＆＃34; a_activityName＆＃34; =＆gt;＆＃34; SetAndLog＆＃34;，＆＃34; offset＆＃34; =＆gt; 498，＆＃34; a_MachineCode＆＃34; =＆gt;＆＃34; CFDM7757＆＃34;，＆ ＃34; input_type＆＃34; =＆gt;＆＃34; log＆＃34;，＆＃34; message＆＃34; =＆gt;＆＃34; 2017-06-05 19：02：46.779 INFO [bwEngThread：In- Memory Process Worker-4] psg.logger - a_applicationName = \＆＃34; PieceProxy \＆＃34;，a_processName = \＆＃34; piece.PieceProxy \＆＃34;，a_jobId = \＆＃34; bw0a10ao \＆ ＃34;，a_processInstanceId = \＆＃34; bw0a10ao \＆＃34;，a_level = \＆＃34; Info \＆＃34;，a_phase = \＆＃34; ProcessStart \＆＃34;，a_activityName = \＆ ＃34; SetAndLog \＆＃34;，a_timeStamp = \＆＃34; 2017-06-05T19：02：46.779 \＆＃34;，a_sessionId = \＆＃34; \＆＃34;，a_sender = \＆＃ 34; PCS \＆＃34;，a_cruid = \＆＃34; 37d7e225-bbe5-425b-8abc-f4b44a5a1560 \＆＃34;，a_MachineCode = \＆＃34; CFDM7757 \＆＃34;，a_correlationId = \＆＃ 34; fa10f \＆＃34;，a_trackingId = \＆＃34; 9d3b8 \＆＃34;，a_message = \＆＃34; START = piece.PieceProxy \＆＃34;＆＃34;，＆＃34; a_phase＆ ＃34; =＆gt;＆＃34; ProcessStart＆＃34;，＆＃34;标签＆＃34; =＆gt; [＆＃34; beats_input_codec_plain_applied＆＃34;，＆＃34; _dateparsefailure＆＃34;，＆＃34; kv_ok＆ ＃34;，＆＃34; t askStarted＆＃34;]，＆＃34; a_processInstanceId＆＃34; =＆gt;＆＃34; bw0a10ao＆＃34;，＆＃34; @ timestamp＆＃34; =＆gt; 2017-06-05T17：02：46.779 Z ，＆＃34; my_index＆＃34; =＆gt;＆＃34; bw_logs＆＃34;，＆＃34; a_timeStamp＆＃34; =＆gt;＆＃34; 2017-06-05T19：02：46.779 ＆＃34;，＆＃34; a_jobId＆＃34; =＆gt;＆＃34; bw0a10ao＆＃34;，＆＃34; a_applicationName＆＃34; =＆gt;＆＃34; PieceProxy＆＃34;，＆＃34; TMS＆ ＃34; =＆gt;＆＃34; 2017-06-05 19：02：46.779＆＃34;}}
NB：
你能解释一下为什么以及@_timestamp改变会发生什么？我不希望它自动完成（我看到很多人都在问这个怎么做），因为@timestamp是一个系统领域。那有什么问题？
这是我的管道：
input {
beats {
port => "5043"
type => json
}
}
filter {
#date {
# match => [ "@timestamp", "ISO8601" ]
# target => "@timestamp"
#}
if "log_bw" in [source] {
grok {
patterns_dir => ["/home/elastic/ELASTIC_NEW/logstash-5.5.0/config/patterns/extrapatterns"]
match => { "message" => "%{CUSTOM_TMS:TMS}\s*%{CUSTOM_LOGLEVEL:log}\s*%{CUSTOM_THREAD:bwthread}\s*%{CUSTOM_LOGGER:logger}-%{CUSTOM_TEXT:text}" }
tag_on_failure => ["no_match"]
}
if "no_match" not in [tags] {
if "Payload for Request is" in [text] {
mutate {
add_field => { "my_index" => "json_request" }
}
grok {
patterns_dir => ["/home/elastic/ELASTIC_NEW/logstash-5.5.0/config/patterns/extrapatterns"]
match => { "text" => "%{CUSTOM_JSON:json_message}" }
}
json {
source => "json_message"
tag_on_failure => ["errore_parser_json"]
target => "json_request"
}
mutate {
remove_field => [ "json_message", "text" ]
}
}
else {
mutate {
add_field => { "my_index" => "bw_logs" }
}
kv {
source => "text"
trim_key => "\s"
field_split => ","
add_tag => [ "kv_ok" ]
}
if "kv_ok" not in [tags] {
drop { }
}
else {
mutate {
remove_field => [ "text" ]
}
if "ProcessStart" in [a_phase] {
mutate {
add_tag => [ "taskStarted" ]
}
}
if "ProcessEnd" in [a_phase] {
mutate {
add_tag => [ "taskTerminated" ]
}
}
date {
match => [ "a_timeStamp", "yyyy'-'MM'-'dd'T'HH:mm:ss.SSS" ]
}
elapsed {
start_tag => "taskStarted"
end_tag => "taskTerminated"
unique_id_field => "a_cruid"
}
}
}
}
}
else {
mutate {
add_field => { "my_index" => "other_products" }
}
}
}
output {
elasticsearch {
index => "%{my_index}"
hosts => ["localhost:9200"]
}
stdout { codec => rubydebug }
file {
path => "/tmp/loggata.tx"
codec => json
}
}
非常感谢，
安德烈
答案 0 :(得分：0)
这是错误（以前测试中的拼写错误）：
date {
match => [ "a_timeStamp", "yyyy'-'MM'-'dd'T'HH:mm:ss.SSS" ]
}
谢谢你们，无论如何！