Question

我正在尝试运行 logstash：latest 容器，但是运行该管道时会抛出错误。

当我直接使用-f“ / path / to / pipeline”运行它时，一切正常。当我从 pipelines.yml 运行它时，它表示管道在第一行第一列的格式有误。

14：39：28.728 [LogStash :: Runner]错误logstash.agent-无法创建管道{：reason =>“在第1行第1列第1列（字节1，预期输入，输入，过滤器，输出之一））之后的“}

注意：在我还没有对第一行发表评论之前，它说错误是在评论之后，因此文件访问和路径绝对可以。

如果有人对我如何或为什么会遇到这种行为有任何想法，我欢迎您提出建议。在底部，我包括我的配置。

pipelines.yml

- pipeline.id: log-pipeline
  path.config: "/etc/logstash/pipelines/log_elastic_write.conf"
  pipeline.workers: 1

log_elastic_write.conf

input { 
        kafka {
                bootstrap_servers => "kafka:9092"
                topics => ["logs"]
                group_id => "logs_write"
                auto_offset_reset => "earliest"
        }
}
filter {
        date {
                match => [ "timestamp" , "yyyy-MM-dd HH:mm:ss.SSSSSS" ]
                timezone => "UTC"
        }
}
filter {
        json {
                source => "message"
        }
}
filter {
        json {
                source => "message"
                target => "raw_message"
        }

        mutate {
                rename => {"@timestamp" => "timestamp_message_received"}
        }

        mutate {
                remove_field => [ "message" ]
                rename => {"[raw_message][timestamp]" => "timestamp_message_sent"}
        }
}
filter {
        date {
                match => ["timestamp_message_sent","yyyy-MM-dd HH:mm:ss.SSS"]
                target => "timestamp_message_sent"
                locale => "en"
        }
}
filter {
        mutate { remove_field => [ "raw_message" ] }
        mutate { remove_field => [ "timestamp" ] }
}
output {
        elasticsearch {
                hosts => "ipelastic:82"
                index => "logging"
        }
}

entrypoint.sh

#!/usr/bin/env bash

echo 'Inspecting file structure'
find /etc/logstash/ -maxdepth 4 | grep -v git | grep -v idea

echo 'Setting up runtime and pipeline configuration files(entrypoint.sh):'
echo "----------        'logstash.yml'        ----------"
envsubst < "${LOGSTASH_DIR_TARGET}/${LOGSTASH_CONFIG_SOURCE}" > "${LOGSTASH_DIR_TARGET}/${LOGSTASH_CONFIG_TARGET}"
rm "${LOGSTASH_DIR_TARGET}/${LOGSTASH_CONFIG_SOURCE}"

echo "----------        'pipelines.yml'        ----------"
envsubst < "${LOGSTASH_DIR_TARGET}/${PIPELINES_CONFIG_SOURCE}" > "${LOGSTASH_DIR_TARGET}/${PIPELINES_CONFIG_TARGET}"
rm "${LOGSTASH_DIR_TARGET}/${PIPELINES_CONFIG_SOURCE}"
less "${LOGSTASH_DIR_TARGET}/${PIPELINES_CONFIG_TARGET}"

echo "---------- pipeline: elasticsearch logs ----------"
envsubst < "${PIPELINE_DIR_TARGET}/${LOG_WRITE_PIPELINE_SOURCE}" > "${PIPELINE_DIR_TARGET}/${LOG_WRITE_PIPELINE_TARGET}"
rm "${PIPELINE_DIR_TARGET}/${LOG_WRITE_PIPELINE_SOURCE}"
less "${PIPELINE_DIR_TARGET}/${LOG_WRITE_PIPELINE_TARGET}"

chmod a+rx "${LOGSTASH_DIR_TARGET}/${LOGSTASH_CONFIG_TARGET}"
chmod a+rx "${LOGSTASH_DIR_TARGET}/${PIPELINES_CONFIG_TARGET}"
chmod a+rx "${PIPELINE_DIR_TARGET}/${LOG_WRITE_PIPELINE_TARGET}"

logstash

Logstash管道无法从pipelines.yml运行

0 个答案: