Question

我试图在 server1 上设置filebeat和logstash，并将数据发送到位于 server2 上的elasticsearch并使用kibana对其进行可视化。

以下是文件信号日志，当我运行 文件信号测试输出 时，结果显示为图像缺陷。

您可以观察到，filbeat根本不收集日志

2020-07-10T07:40:14.852Z    DEBUG   [input] input/input.go:141  Run input
2020-07-10T07:40:14.852Z    DEBUG   [input] log/input.go:191    Start next scan
2020-07-10T07:40:14.852Z    DEBUG   [input] log/input.go:212    input states cleaned up. Before: 0, After: 0, Pending: 0
2020-07-10T07:40:24.853Z    DEBUG   [input] input/input.go:141  Run input
2020-07-10T07:40:24.853Z    DEBUG   [input] log/input.go:191    Start next scan
2020-07-10T07:40:24.853Z    DEBUG   [input] log/input.go:212    input states cleaned up. Before: 0, After: 0, Pending: 0
2020-07-10T07:40:34.853Z    DEBUG   [input] input/input.go:141  Run input
2020-07-10T07:40:34.853Z    DEBUG   [input] log/input.go:191    Start next scan
2020-07-10T07:40:34.853Z    DEBUG   [input] log/input.go:212    input states cleaned up. Before: 0, After: 0, Pending: 0
2020-07-10T07:40:44.828Z    INFO    [monitoring]    log/log.go:145  Non-zero metrics in the last 30s    {"monitoring": {"metrics": {"beat":{"cpu":{"system":{"ticks":20},"total":{"ticks":170,"time":{"ms":4},"value":170},"user":{"ticks":150,"time":{"ms":4}}},"handles":{"limit":{"hard":1048576,"soft":1024},"open":7},"info":{"ephemeral_id":"4f97f60d-b9f4-451c-b9f2-1935988798b1","uptime":{"ms":840027}},"memstats":{"gc_next":10220512,"memory_alloc":5959576,"memory_total":24826152},"runtime":{"goroutines":21}},"filebeat":{"harvester":{"open_files":0,"running":0}},"libbeat":{"config":{"module":{"running":0}},"pipeline":{"clients":1,"events":{"active":0}}},"registrar":{"states":{"current":0}},"system":{"load":{"1":0.03,"15":0.08,"5":0.06,"norm":{"1":0.015,"15":0.04,"5":0.03}}}}}}
2020-07-10T07:40:44.853Z    DEBUG   [input] input/input.go:141  Run input
2020-07-10T07:40:44.856Z    DEBUG   [input] log/input.go:191    Start next scan
2020-07-10T07:40:44.856Z    DEBUG   [input] log/input.go:212    input states cleaned up. Before: 0, After: 0, Pending: 0
2020-07-10T07:40:54.856Z    DEBUG   [input] input/input.go:141  Run input
2020-07-10T07:40:54.856Z    DEBUG   [input] log/input.go:191    Start next scan
2020-07-10T07:40:54.856Z    DEBUG   [input] log/input.go:212    input states cleaned up. Before: 0, After: 0, Pending: 0
2020-07-10T07:41:04.856Z    DEBUG   [input] input/input.go:141  Run input
2020-07-10T07:41:04.856Z    DEBUG   [input] log/input.go:191    Start next scan
2020-07-10T07:41:04.856Z    DEBUG   [input] log/input.go:212    input states cleaned up. Before: 0, After: 0, Pending: 0
2020-07-10T07:41:14.828Z    INFO    [monitoring]    log/log.go:145  Non-zero metrics in the last 30s    {"monitoring": {"metrics": {"beat":{"cpu":{"system":{"ticks":20},"total":{"ticks":180,"time":{"ms":8},"value":180},"user":{"ticks":160,"time":{"ms":8}}},"handles":{"limit":{"hard":1048576,"soft":1024},"open":7},"info":{"ephemeral_id":"4f97f60d-b9f4-451c-b9f2-1935988798b1","uptime":{"ms":870027}},"memstats":{"gc_next":10216448,"memory_alloc":5117872,"memory_total":25101640},"runtime":{"goroutines":21}},"filebeat":{"harvester":{"open_files":0,"running":0}},"libbeat":{"config":{"module":{"running":0}},"pipeline":{"clients":1,"events":{"active":0}}},"registrar":{"states":{"current":0}},"system":{"load":{"1":0.02,"15":0.08,"5":0.05,"norm":{"1":0.01,"15":0.04,"5":0.025}}}}}}
2020-07-10T07:41:14.856Z    DEBUG   [input] input/input.go:141  Run input
2020-07-10T07:41:14.856Z    DEBUG   [input] log/input.go:191    Start next scan
2020-07-10T07:41:14.856Z    DEBUG   [input] log/input.go:212    input states cleaned up. Before: 0, After: 0, Pending: 0
2020-07-10T07:41:24.856Z    DEBUG   [input] input/input.go:141  Run input
2020-07-10T07:41:24.856Z    DEBUG   [input] log/input.go:191    Start next scan
2020-07-10T07:41:24.856Z    DEBUG   [input] log/input.go:212    input states cleaned up. Before: 0, After: 0, Pending: 0
2020-07-10T07:41:34.857Z    DEBUG   [input] input/input.go:141  Run input
2020-07-10T07:41:34.857Z    DEBUG   [input] log/input.go:191    Start next scan
2020-07-10T07:41:34.857Z    DEBUG   [input] log/input.go:212    input states cleaned up. Before: 0, After: 0, Pending: 0

filebeat.yml

filebeat.inputs:
- type: log
  enabled: true
  paths:
    - /home/mahesh/Documents/refactor/nomi/unity/media/*.log

output.logstash:
  enabled: true
  hosts: ["localhost:5044"]

logstash.conf

input {
beats {
    port => 5044
    ssl => false
  }
}

filter {
  grok {
    match => { "message" => "%{TIMESTAMP_ISO8601:timestamp}] %{LOGLEVEL:loglevel}\|%{GREEDYDATA:module}\|%{GREEDYDATA:content}" }
  }
  date {
    locale => "en"
    match => [ "timestamp", "YYYY-MM-dd HH:mm:ss"]
    target => "@timestamp"
    timezone => "America/New_York"
  }
}

output {
  elasticsearch {
    hosts => "elk_server_ip:9200"
    manage_template => false
    index => "blend_test" 
  }
  stdout { codec => rubydebug { metadata => true } }
}

Answer 1

由于以下原因，文件拍未捕获日志的原因：

注册表文件夹s = v*t中的

data.json可能不为空，这意味着日志文件已在第一次或连续扫描时进行了解析。

您可以删除 data.json 文件中的内容，然后重新启动文件信号。

希望这可以解决您的问题。

Filebeat无法收集日志

1 个答案: