我的要求是将filebeat配置为能够发送日志以进行弹性搜索,文件beat的源是docker容器日志。
我正在使用docker安装filebeat,下面提到的是我用于配置的dockerfile.filebeat.yml和docker-compose文件。
DockerFile:
FROM docker.elastic.co/beats/filebeat:7.2.1
# Copy our custom configuration file
COPY filebeat.yml /usr/share/filebeat/filebeat.yml
USER root
# Create a directory to map volume with all docker log files
RUN mkdir /usr/share/filebeat/dockerlogs
RUN chown -R root /usr/share/filebeat/
RUN chmod -R go-w /usr/share/filebeat/
filebeat.yml
#filebeat.modules:
#- module: system
# syslog:
# enabled: true
#auth:
#enabled: true
#- module: auditd
#log:
# Does not look like Auditd is supported in Alpine linux: https://github.com/linuxkit/linuxkit/issues/52
#enabled: false
filebeat.inputs:
- type: docker
enabled: true
containers:
path: "/var/lib/docker/containers"
stream: all # can be all, stdout or stderr
ids:
- '*'
# exclude_lines: ["^\\s+[\\-`('.|_]"] # drop asciiart lines
# multiline.pattern: "^\t|^[[:space:]]+(at|...)|^Caused by:"
# multiline.match: after
#========================== Filebeat autodiscover ==============================
# See this URL on how to run Apache2 Filebeat module: # https://www.elastic.co/guide/en/beats/filebeat/current/running-on-docker.html
#filebeat.autodiscover:
# providers:
# - type: docker
# https://www.elastic.co/guide/en/beats/filebeat/current/configuration-autodiscover-hints.html
# This URL alos contains instructions on multi-line logs
# hints.enabled: true
#================================ Processors ===================================
processors:
#- add_cloud_metadata: ~
- add_docker_metadata: ~
- add_locale:
format: offset
- add_host_metadata:
netinfo.enabled: true
#========================== Elasticsearch output ===============================
output.elasticsearch:
hosts: ["http://192.168.241.40:9200"]
docker-compose
version: '2'
services:
filebeat:
hostname: filebeat
# ** Here to build the image, you need to specify your own docker hub account :
image: filebeat/img-1
volumes:
# needed to persist filebeat tracking data :
- "filebeat_data:/usr/share/filebeat/data:rw"
# needed to access all docker logs (read only) :
- "/var/lib/docker/containers:/usr/share/dockerlogs/data:ro"
# needed to access additional informations about containers
- "/var/run/docker.sock:/var/run/docker.sock"
volumes:
# create a persistent volume for Filebeat
filebeat_data:
使用此配置,我能够在我的计算机上安装文件拍子,但是当我在主机上运行服务时,我无法使用文件拍子捕获生成的日志并将其发送到Elastic-Search。
我正在做的可能是什么错误,感谢您的帮助
注意:ElasticSearch <7.2.1>和kibana <7.2.1>已安装在同一台计算机上,我可以通过192.168.241.40:9200打开它们
答案 0 :(得分:1)
filebeat 版本:7.12.0 当您使用 docker 日志时,您需要配置自动发现。
filebeat.yml
# # =========================== Filebeat autodiscover ============================
filebeat.autodiscover:
providers:
- type: docker
templates:
- condition:
contains:
docker.container.image:<your_label_condition>
config:
- type: container
paths:
- "/var/lib/docker/containers/${data.docker.container.id}/*.log"
exclude_lines: ["^\\s+[\\-`('.|_]"]
filebeat.shutdown_timeout: 5s #optional
# ------------------------------- Console Output -------------------------------
output.console:
enabled: true
codec.json:
pretty: true
escape_html: false
logging.metrics.enabled: false
在发送到 logstash 之前,我使用控制台输出来验证一切正常。