Question

我正在使用Logstash读取日志文件。这是文件：

配置文件：

input { 
file{
    path => "/home/cdot/Desktop/auth_log"
    start_position => beginning
}
}

filter{
grok{

match => ["message", "%{TIMESTAMP_ISO8601: timestamp} %{HOSTNAME: server-name} %{WORD: action}: %{WORD: machine}(%{GREEDYDATA: command}):%{GREEDYDATA:logline}"]
}
}

output {
    elasticsearch { host => localhost }
    stdout { codec => rubydebug }
}

输出：

Using milestone 2 input plugin 'file'. This plugin should be stable, but if you see strange behavior, please let us know! For more information on plugin milestones, see http://logstash.net/docs/1.4.1/plugin-milestones {:level=>:warn}

我没有得到任何输出。我的日志文件包含以下形式的行：

2014-05-09T04:02:08+05:30 bx920as1 runuser: pam_unix(runuser-l:session): session opened for user cyrus by (uid=0)
2014-05-26T04:02:09+05:30 bx920as1 runuser: pam_unix(runuser-l:session): session closed for user cyrus

Plz帮助。

修改

添加行

start_position => beginning
    sincedb_path => "/dev/null"

输入我得到以下输出：

{
       "message" => "2014-05-26T04:02:09+05:30 bx920as1 runuser: pam_unix(runuser-l:session): session opened for user cyrus by (uid=0)",
      "@version" => "1",
    "@timestamp" => "2014-05-27T03:59:26.773Z",
          "host" => "cdot-HP-Pro-3330-MT",
          "path" => "/home/cdot/Desktop/auth_log",
       "logline" => " session opened for user cyrus by (uid=0)"
}
{
       "message" => "2014-05-26T04:02:09+05:30 bx920as1 runuser: pam_unix(runuser-l:session): session closed for user cyrus",
      "@version" => "1",
    "@timestamp" => "2014-05-27T03:59:26.774Z",
          "host" => "cdot-HP-Pro-3330-MT",
          "path" => "/home/cdot/Desktop/auth_log",
       "logline" => " session closed for user cyrus"
}
{
       "message" => "",
      "@version" => "1",
    "@timestamp" => "2014-05-27T03:59:26.774Z",
          "host" => "cdot-HP-Pro-3330-MT",
          "path" => "/home/cdot/Desktop/auth_log",
          "tags" => [
        [0] "_grokparsefailure"
    ]
}

因此，仅捕获日志并且不会匹配休息字段。有什么想法吗？

Answer 1

Logstash文件输入将跟踪受监控日志文件的当前位置，并将当前位置保存到sincedb，默认路径是您的主目录。请参阅here

因此，start_position => beginning仅在您第一次开始监控文件时生效。之后，logstash将从保存在sincedb中的位置开始。

因此，如果您始终想要从第一行读取日志，请将此配置添加到input文件

sincedb_path => "/dev/null"

或

删除主目录中的所有.sincedb文件。您还可以在启动logstash后将日志输入到监视器日志文件中。

Answer 2

<强>解决：问题是由于其他标识符的错误表达（因此它们没有显示）和logline表达式正确（因此显示）。

使用里程碑2输入插件＆＃39;文件＆＃39;？

2 个答案: