Question

新的logstash，非常享受它。

尝试解析包含时间戳的CSV文件。想要解析时间戳并将其用作@timestamp字段。

我的CSV输入样本

input {
    stdin {}
}

filter {
    # filter the input by csv (i.e. comma-separated-value)
    csv {
        columns => [
            "Job ID",
            "Server Name",
            "Status Code",
            "Job Type",
            "Client Name",
            "Start Time",
            "End Time"
        ]
    }
    # parse the start time to create a real date
    date {
        # Examples of times in this log file
        # "May 29, 2015 10:00:01 PM"
        # "May 9, 2015 4:47:23 AM"
        match => [ "End Time",
                   "MMM dd, YYYY HH:mm:ss aa",
                   "MMM  d, YYYY HH:mm:ss aa" ]
    }
}

# send the output to stdout, using the rubydebug codec
# rubydedug uses the Ruby Awesome Print library
output {
    stdout { codec => rubydebug }
}

我的输入样本

108628,anmuswcnbu01,1,Backup,anmuswcrfax01.na.jnj.com,"May 29, 2015 10:00:01 PM","May 30, 2015 6:21:29 AM"
108629,anmuswcnbu01,1,Backup,anmuswcapps01.na.jnj.com,"May 29, 2015 10:00:01 PM","May 9, 2015 10:51:39 pm"
108630,anmuswcnbu01,1,Backup,anmuswcapps03.na.jnj.com,"May 29, 2015 10:00:01 PM","May 29, 2015 9:31:19 PM"

我的输出样本

Logstash startup completed
{
        "message" => [
        [0] "108628,anmuswcnbu01,1,Backup,anmuswcrfax01.na.jnj.com,\"May 29, 2015 10:00:01 PM\",\"May 30, 2015 6:21:29 AM\"\r"
    ],
       "@version" => "1",
     "@timestamp" => "2015-05-30T06:21:29.000Z",
           "host" => "ip-172-31-34-14",
         "Job ID" => "108628",
    "Server Name" => "anmuswcnbu01",
    "Status Code" => "1",
       "Job Type" => "Backup",
    "Client Name" => "anmuswcrfax01.na.jnj.com",
     "Start Time" => "May 29, 2015 10:00:01 PM",
       "End Time" => "May 30, 2015 6:21:29 AM"
}
{
        "message" => [
        [0] "108629,anmuswcnbu01,1,Backup,anmuswcapps01.na.jnj.com,\"May 29, 2015 10:00:01 PM\",\"May 9, 2015 10:51:39 pm\"\r"
    ],
       "@version" => "1",
     "@timestamp" => "2015-05-09T10:51:39.000Z",
           "host" => "ip-172-31-34-14",
         "Job ID" => "108629",
    "Server Name" => "anmuswcnbu01",
    "Status Code" => "1",
       "Job Type" => "Backup",
    "Client Name" => "anmuswcapps01.na.jnj.com",
     "Start Time" => "May 29, 2015 10:00:01 PM",
       "End Time" => "May 9, 2015 10:51:39 pm"
}
{
        "message" => [
        [0] "108630,anmuswcnbu01,1,Backup,anmuswcapps03.na.jnj.com,\"May 29, 2015 10:00:01 PM\",\"May 29, 2015 9:31:19 PM\"\r"
    ],
       "@version" => "1",
     "@timestamp" => "2015-05-29T09:31:19.000Z",
           "host" => "ip-172-31-34-14",
         "Job ID" => "108630",
    "Server Name" => "anmuswcnbu01",
    "Status Code" => "1",
       "Job Type" => "Backup",
    "Client Name" => "anmuswcapps03.na.jnj.com",
     "Start Time" => "May 29, 2015 10:00:01 PM",
       "End Time" => "May 29, 2015 9:31:19 PM"
}
Logstash shutdown completed

例如，在最后（第3）行数据中，而不是：

"@timestamp" => "2015-05-29T09:31:19.000Z",

我觉得我应该得到

"@timestamp" => "2015-05-29T21:31:19.000Z",

我可以说，日期过滤器似乎忽略了我的“半天”语法

 match => [ "End Time",
                   "MMM dd, YYYY HH:mm:ss aa",
                   "MMM  d, YYYY HH:mm:ss aa" ]

新的logstash，所以想知道我做错了什么？

-Chad

Answer 1

date过滤器使用与Joda-Time兼容的格式。

引用Joda符号表的一部分：

 Symbol  Meaning                      Presentation  Examples
 ------  -------                      ------------  -------   
 a       halfday of day               text          PM
 K       hour of halfday (0~11)       number        0
 h       clockhour of halfday (1~12)  number        12

 H       hour of day (0~23)           number        0
 k       clockhour of day (1~24)      number        24
 m       minute of hour               number        30
 s       second of minute             number        55
 S       fraction of second           number        978

这很容易被忽视，但在你的情况下，半天时间是KK而不是HH。

logstash解析时间戳半天上午/下午

1 个答案: