Question

我一直在使用logstash来读取一些数据库恢复日志。以下是一些样本记录。

07/08/2016  6:33:22.50: START restore database                      
SQL2540W  Restore is successful, however a warning "2539" was encountered 
during Database Restore while processing in No Interrupt mode.
07/08/2016  6:33:28.93: END restore database                        
SQL4406W  The DB2 Administration Server was started successfully.
07/08/2016  6:35:35.29: END restart server                          
connect reset
DB20000I  The SQL command completed successfully.
07/08/2016  6:35:38.48: END p:\s6\source\system\CMD\res_uw.cmd

以下是我的conf文件的过滤器部分。

if ([message] =~ /Backup successful/){
    grok{
        match => {"message" => ['%{GREEDYDATA:Message}'] }
    }
    mutate {
        add_tag => "send_to_es"
        add_field => {"Timestamp" => "%{GREEDYDATA:DATETIME}"}
    }
}
if ([message] =~ /warning "2539"/){
    grok{
        match => {"message" => ['%{GREEDYDATA:Message}'] }
    }
    mutate {
        add_tag => "send_to_es"
        add_field => {"Timestamp" => "%{GREEDYDATA:DATETIME}"}
    }
}
if ([message] =~ /(END p:|END P:)/){
    grok{
        match => {"message" => ['%{GREEDYDATA:DATETIME}:%{SPACE}END%{SPACE}%{GREEDYDATA:Mis}'] }
        remove_field => "%{GREEDYDATA:Mis}"
    }
    mutate {
        add_tag => "send_to_es"
    }
}

我想添加数据＆＃34; DATETIME＆＃34;从我的记录的最后一行提取消息到同时索引的其他消息。但是，它无法成功添加该字段。输出将变为

      "message": "SQL2540W  Restore is successful, however a warning \"2539\" was encountered \r\r",
      "@version": "1",
      "@timestamp": "2016-07-12T02:28:52.337Z",
      "path": "C:/CIGNA/hkiapp67_db_restore/res_uw.log",
      "host": "SIMSPad",
      "type": "txt",
      "Message": "SQL2540W  Restore is successful, however a warning \"2539\" was encountered \r\r",
      "Timestamp": "%{GREEDYDATA:DATETIME}",
      "tags": [
        "send_to_es"
      ]

我怎么解决这个问题？

Answer 1

Logstash在收到一行时，不知道任何其他行。您必须使用多行编解码器/过滤器将所需的所有行重新组合为带有日期的行。然后使用grok过滤器提取日期并将其添加到文档中。

多行编解码器/过滤器的配置如下所示：

multiline {
  pattern => "%{DATE}"
  negate => "true"
  what => "next"
}

这样，所有不以模式DATE开头的行都将与下一行连接。

Logstash无法添加字段？

1 个答案: