Question

我只是ELK的新手，并尝试对此进行一些测试，我可以运行一些测试，但我正在尝试使用final class ReplaceBundleServicePass implements CompilerPassInterface { public function process(ContainerBuilder $container) { $serviceDefinition = $container->getDefinition('bundle_service'); $serviceDefinition->setClass(MyOwnClass::class); } }＆amp; grok到remoev＆amp;从我的syslog输出中替换一些feilds我正在进入下面的错误..

mutate

以下是我的配置文件....

21:58:47.976 [LogStash::Runner] ERROR logstash.agent - Cannot create pipeline {:reason=>"Expected one of #, {, ,, ] at line 21, column 9 (byte 496) after filter {\n  if [type] == \"syslog\" {\n    grok {\n      match => { \"message\" => \"%{SYSLOGTIMESTAMP:syslog_timestamp} %{SYSLOGHOST:hostname} %{DATA:program}(?:\\[%{POSINT:pid}\\])?: %{GREEDYDATA:syslog_message}\" }\n    }\n    date {\n      match => [ \"syslog_timestamp\", \"MMM  d HH:mm:ss\", \"MMM dd HH:mm:ss\" ]\n    }\n    mutate {\n      remove_field => [\n        \"message\",\n        \"pid\",\n        \"port\"\n        "}

请建议我做错了什么，并帮助理解删除＆amp;替换lagstash的函数..

PS：我的ELK版本是5.4

Answer 1

你发布的Config有很多语法错误，logsatsh有它自己的配置语言，并期望配置文件遵守规则。这个link具有完整的logstash配置语言参考。

我对您的配置文件进行了一些更正并发布到此处，添加了我对配置文件本身错误的评论和解释

input 
{
    file 
    {
        path => [ "/scratch/rsyslog/*/messages.log" ]
        type => "syslog"
    }
}

filter 
{
    if [type] == "syslog" 
    {
        grok 
        {
            match => { "message" => "%{SYSLOGTIMESTAMP:syslog_timestamp} %{SYSLOGHOST:hostname} %{DATA:program}(?:\[%{POSINT:pid}\])?: %{GREEDYDATA:syslog_message}" }
        }

    date 
    {
        match => [ "syslog_timestamp", "MMM  d HH:mm:ss", "MMM dd HH:mm:ss" ]
    }

    # Have merged it with the remove_field option below
    #mutate {
    #  remove_field => [
    #    "message",
    #    "pid",
    #    "port",
    #    "_grokparsefailure"
    #  ]
    #}

    mutate 
    {

        # The replace option only accept hash data type which has a syntax as below 
        # For more details visit the below link
        # https://www.elastic.co/guide/en/logstash/current/plugins-filters-mutate.html#plugins-filters-mutate-replace
        replace => {
            "@source_host" => "%{allLogs_hostname}" 
            "@message" => "%{allLogs_message}"
        }
    }

    mutate 
    {
        # Mutate does not have remove option i guess your intention is to remove the event field
        # hence used remove_field option here
        # The remove_filed option only accepts arary as value type as shown below
        # For details read the below link
        # https://www.elastic.co/guide/en/logstash/current/plugins-filters-mutate.html#plugins-filters-mutate-remove_field
        remove_field => [
            "message",
            "pid",
            "port",
            "_grokparsefailure",
            "allLogs_hostname",
            "syslog_message",
            "syslog_timestamp"
        ]
    }
  }
}

output 
{
    if [type] == "syslog" 
    {
        elasticsearch 
        {
            # The Hosts option only takes uri as a value type , originally you have provided string as it's value type
            # For more info please read the below link
            #https://www.elastic.co/guide/en/logstash/current/plugins-outputs-elasticsearch.html#plugins-outputs-elasticsearch-hosts
            hosts => ["localhost:9200"]
            index => "%{type}-%{+YYYY.MM.dd}"
        }
    }
}

您可以使用logstash命令行选项测试配置文件在语法上是否正确-t此选项将测试并报告配置文件在语法上是否正确

bin\logstash -f 'path-to-your-config-file' -t

如有任何澄清，请通知我

Answer 2

你必须在＆＃34; port＆＃34;之后添加一个逗号。在您的logstash配置文件中。

   mutate {
      remove_field => [
        "message",
        "pid",
        "port",
        "_grokparsefailure"
      ]
    }

无法摄取syslog-logstash.conf以进行删除＆amp;替换功能

2 个答案: