Question

input {
  file {
    path => "C:\Data\data.log"
    start_position => "beginning"
    sincedb_path => "NUL"

  }
}
filter {
    if [type] == "apache" {
        grok {
                    match => ["message", "%{COMBINEDAPACHELOG} "]
        }
    }mutate{
           remove_field => ["@timestamp"]
           remove_field => ["host"]
           remove_field => ["@version"]
           remove_field => ["path"]
    }   
}

output {
    elasticsearch{
    hosts => "localhost:9200"
    index => "logdata2"
    document_type => "logs"
    }
    stdout {codec => rubydebug}
}

以下是我遇到的问题：

我想挑出一些单词，但从来没有能够把它弄好。

我想要的只是获取一个带有时间戳的字符串，该字符串位于消息字符串中。还有另一个词，比如OrderCreated。

是否可以通过这种方式从消息字段中选择特定的字符串/单词？

解剖工作得很好，但现在我遇到了一个我以前没有遇到过的问题。

dissect filter 
input {
  file {
    path => "C:\Data\Logs\testrunning.log"
    start_position => "beginning"
    sincedb_path => "NUL"

  }
}
  filter {
    dissect {
      mapping => {
        "message" => "%{ts} %{+ts} %{+ts} %{src} %{} : %{msg}"
      }
    }mutate { remove_field => "@timestamp" 
    remove_field => "pid"
    remove_field => "prog"
    remove_field => "@version"
    remove_field => "host"
    remove_field => "path"
    remove_field => "src"
  } 
}
output {
    elasticsearch{
    hosts => "localhost:9200"
    index => "logdata12"
    document_type => "logs"
    }
    stdout {codec => rubydebug}
}

输出如下。这对我来说是新的，以前没有的“\ r \ n”部分..这对任何人来说都很熟悉吗？我该如何解决这个问题？

{
    "message" => "General 2018-05-17 15:47:33.149 : StatusInformationSomeData.Unsubscribe() \r",
        "msg" => "StatusInformationSomeData.Unsubscribe() \r",
         "ts" => "General 2018-05-17 15:47:33.149"
}
{
    "message" => "\r",
        "msg" => "\r",
         "ts" => "  "
}

Answer 1

You can use dissect{}. As an example, if you have a log line such as:

198.41.30.203 - - [21/May/2018:14:36:35 -0500] "GET /tag/eclipse/feed/ HTTP/1.1" 404 5232 "-" "UniversalFeedParser/4.2-pre-308-svn +http://feedparser.org/"

Your dissect could be something like this, and you can even convert datatypes. The performance of dissect is much better than grok:

dissect {
    mapping => {
        "message" => '%{source_ip} %{} %{username} [%{raw_timestamp}] "%{http_verb} %{http_path} %{http_version}" %{http_response} %{http_bytes} "%{site}" "%{useragent}"'
        convert_datatype => {
            http_bytes => "int"
        }
    } 
}

grok过滤器从消息字段

1 个答案: