Question

我已经设置了logstash来读取XML文件。它确实这样做，但它不会输出到stdout直到我退出应用程序（ctrl + c）。然后它打印出它所具有的正确结构：

.conf文件：

input {
    file {
        path => "C:/Projects/Python Projects/SolAdmin/SolAdmin/ClientConnectionsWide_*.xml"
        start_position => beginning
        ignore_older => 0
        sincedb_path => "C:/Projects/Elastic Stack/logstash-2.3.4/logstash-2.3.4/sincedb"
        codec => multiline {
            pattern => "^<rpc-reply.*\>"
            negate => true
            what => "previous"
        }
    }
}

filter {
    grok {
      match => { "message" => "%{GREEDYDATA:xmldata}" }
    }
    xml {
        source => "xmldata"
        store_xml => "false"
        xpath => [
            "rpc-reply/rpc/show/client/client/connection/foreign-address/text()", "client_ip",
            "rpc-reply/rpc/show/client/client/connection/round-trip-time-smooth-us/text()", "smooth_round_trip"
        ]
    }
}

output {
        stdout {
        codec => rubydebug 
    }
}

.XML：

<rpc-reply semp-version="soltr/7_1_1">
  <rpc>
    <show>
      <client>
        <client>
          <name>BSPFODS0914/6548/#00000001</name>
          <message-vpn>fod_prod</message-vpn>
          <connection>
            <protocol>tcp</protocol>
            <is-zip>false</is-zip>
            <is-ssl>false</is-ssl>
            <receive-queue-bytes>0</receive-queue-bytes>
            <receive-queue-segments>0</receive-queue-segments>
            <send-queue-bytes>0</send-queue-bytes>
            <send-queue-segments>0</send-queue-segments>
            <local-address>10.137.4.211:55555</local-address>
            <foreign-address>10.134.100.72:52974</foreign-address>
            <state>ESTABLISHED</state>
            <maximum-segment-size>1460</maximum-segment-size>
            <bytes-sent-32bits>52206343</bytes-sent-32bits>
            <bytes-received-32bits>6779578</bytes-received-32bits>
            <retransmit-time-ms>278.567</retransmit-time-ms>
            <round-trip-time-smooth-us>737.83</round-trip-time-smooth-us>
            <round-trip-time-minimum-us>584.758</round-trip-time-minimum-us>
            <round-trip-time-variance-us>60.759</round-trip-time-variance-us>
            <advertised-window-size>262144</advertised-window-size>
            <transmit-window-size>149116</transmit-window-size>
            <bandwidth-window-size>22617</bandwidth-window-size>
            <congestion-window-size>131072</congestion-window-size>
            <slow-start-threshold-size>262144</slow-start-threshold-size>
            <segments-received-out-of-order>0</segments-received-out-of-order>
            <fast-retransmits>0</fast-retransmits>
            <timed-retransmits>60</timed-retransmits>
            <connection-uptime-s>16886</connection-uptime-s>
            <blocked-cycles-percent>0</blocked-cycles-percent>
            <interface>1/6/1</interface>
          </connection>
        </client>
      </client>
    </show>
  </rpc>
<execute-result code="ok"/>
</rpc-reply>

结果：

C:\Projects\Elastic Stack\logstash-2.3.4\logstash-2.3.4>bin\logstash -f config\Config.conf
Using JAVA_HOME=C:\Program Files (x86)\Java\jre1.8.0_91 retrieved from C:\ProgramData\Oracle\java\javapath\java.exe
io/console not supported; tty will not be manipulated
Settings: Default pipeline workers: 8
Pipeline main started
←[33mSIGINT received. Shutting down the agent. {:level=>:warn}←[0m
stopping pipeline {:id=>"main"}
{
           "@timestamp" => "2016-07-26T15:27:43.866Z",
              "message" => "<rpc-reply semp-version=\"soltr/7_1_1\">\r\n  <rpc>\r\n    <show>\r\n      <client>\r\n        <client>\r\n          <name>BSPFODS0
             "@version" => "1",
                 "tags" => [
        [0] "multiline"
    ],
                 "path" => "C:/Projects/Python Projects/SolAdmin/SolAdmin/ClientConnectionsWide_20160726-131028.xml",
                 "host" => "CP-FOD-110805",
              "xmldata" => "<rpc-reply semp-version=\"soltr/7_1_1\">\r\n  <rpc>\r\n    <show>\r\n      <client>\r\n        <client>\r\n          <name>BSPFODS0
            "client_ip" => [
        [0] "10.134.100.72:52974"
    ],
    "smooth_round_trip" => [
        [0] "737.83"
    ]
}

Answer 1

codec-multiline一直在等待下一行检查它是否属于上一行。在what = "next"的情况下，它一直等待下一行将当前行附加到它。所以基本上它总是在等待下一行，因此你的日志中的最后一行永远不会被刷新到输出。

此问题已在此处报告：https://github.com/elastic/logstash/issues/4567

此问题的解决方案已在Logstash＆gt; = 2.1中定义。 auto_flush_interval的{{1}}属性可用于在一定时间间隔后刷新输出。

您可以按如下方式修改配置：

codec-multiline

在配置中进行此修改后，logstash现在将等待5秒，然后它将清除所有剩余的输出行。

codec => multiline { pattern => "^<rpc-reply.*\>" negate => true what => "previous" auto_flush_interval => 5 }没有默认值，因此您必须明确指定它以从Logstash获取此行为。

我在auto_flush_interval上测试了它，它对我有用。希望它也能帮到你！

参考：codec-multiline-auto_flush_interval

在CTRL-C之前，logstash不打印输出

1 个答案: