Question

我正在尝试使用systemd在运行ubuntu 16.04的aws ec2实例上运行logstash。我在机器上安装了heroku工具带。正常运行管道（通过bin / logstash.bat）工作正常并且事件被摄取（但是几分钟后得到“请求超时”错误并且管道停止，这是一个单独的问题）。

但是当我尝试在systemd上运行服务时，我会收到错误，不确定这两种类型的错误是否相关。第一个是SSL错误：

错误：没有密码匹配（OpenSSL :: SSL :: SSLError）

[2017-02-15T13：08：43,437] [错误] [logstash.pipeline]一个插件   有一个不可恢复的错误。将重启此插件。插入：   “XXXXXX”，   编解码器=＆gt;“中^％{TIMESTAMP_ISO8601}   ％{WORD} \ [\ w +（\。\ d +）？\] :( \ s {3，} | \}）“，what =＆gt;”previous“，   id =＆gt;“032c3b317ae49982945ec7e8fbf11224be98f237-3”，enable_metric =＆gt; true，   negate =＆gt; false，charset =＆gt;“UTF-8”，multiline_tag =＆gt;“multiline”，   max_lines =＆gt; 500，max_bytes =＆gt; 10485760＆gt;，   id =＆gt;“032c3b317ae49982945ec7e8fbf11224be98f237-4”，enable_metric =＆gt; true＆gt;

第二个是heroku toolbelt似乎在提示凭据：

Feb 15 13:08:43 ip-10-0-1-216 logstash [4402]：输入你的Heroku   证书。

Feb 15 13:08:43 ip-10-0-1-216 logstash [4402]：电子邮件：   密码（输入将被隐藏）：

我的logstash配置：

input {
    heroku {
        app => "xxx-1"
        codec => multiline {
            pattern => "^%{TIMESTAMP_ISO8601} %{WORD}\[\w+(\.\d+)?\]:(\s{3,}| \})"
            what => "previous"
        }
    }
    heroku {
        app => "xxx-2"
        codec => multiline {
            pattern => "^%{TIMESTAMP_ISO8601} %{WORD}\[\w+(\.\d+)?\]:(\s{3,}| \})"
            what => "previous"
        }
    }
    heroku {
        app => "xxx-3"
        codec => multiline {
            pattern => "^%{TIMESTAMP_ISO8601} %{WORD}\[\w+(\.\d+)?\]:(\s{3,}| \})"
            what => "previous"
        }
    }
    heroku {
        app => "xxx-4"
        codec => multiline {
            pattern => "^%{TIMESTAMP_ISO8601} %{WORD}\[\w+(\.\d+)?\]:(\s{3,}| \})"
            what => "previous"
        }
    }
}

filter {
  grok {
    break_on_match => true
    patterns_dir => ["./grok_patterns"]
    match => { "message" => [
        "^%{TIMESTAMP_ISO8601:timestamp} %{WORD:heroku_source}\[%{DYNO:dyno}\]: %{LEVEL:level}: HTTP %{OPT_NOT_SPACE_COMMA:organization}, %{OPT_NOT_COMMA:user}, %{OPT_NOT_COMMA:device}, %{WORD:method} %{ENDPOINT:endpoint}%{QUERY:query} \[%{INT:responseCode:int}\].*? \(p%{INT:nodeProcess:int}\) \(%{INT:responseTime:int}ms\).*$",
        "^%{TIMESTAMP_ISO8601:timestamp} %{WORD:heroku}\[%{WORD:component}\]: at=\w+ method=%{WORD:method} path=\"%{ENDPOINT:endpoint}\??%{QUERY:query}\" .*?fwd=\"%{IP:site_ip}\" dyno=%{DYNO:dyno} .*?service=%{INT:responseTime:int}ms status=%{INT:responseCode:int} bytes=%{INT:sizeBytes:int}.*?$",
        "^%{TIMESTAMP_ISO8601:timestamp} %{WORD:heroku_source}\[%{DYNO:dyno}\]: (?<data>.*)"
    ] }
    add_field => { "endpoint_template" => "%{endpoint}" }
  }
  mutate {
    gsub => ["endpoint_template", "[0-9a-f]{24}", "ID"]
    add_field => { "type" => "heroku" }
  }
  if ![heroku_source] {
    geoip {
        source => "site_ip"
    }
    mutate {
        add_field => { "heroku_source" => "heroku" }
    }
  }
}

output {
    elasticsearch {
        hosts => [ "aws-es-endpoint:443" ]
        ssl => true        
    }
}

（我确信它可以改进）

我尝试以root身份运行服务，但结果是一样的。只是为了澄清，这有效：

/usr/share/logstash/bin/logstash --path.settings /etc/logstash/

虽然这不是：

sudo systemctl start logstash

这是procedures on elastic之后的logstash 5.2.1的全新安装。 Systemd也根据their procedures运行，因此它执行与我手动执行相同的命令。 cat logstash.service输出：

[Unit]
Description=logstash

[Service]
Type=simple
User=logstash
Group=logstash
# Load env vars from /etc/default/ and /etc/sysconfig/ if they exist.
# Prefixing the path with '-' makes it try to load, but if the file doesn't
# exist, it continues onward.
EnvironmentFile=-/etc/default/logstash
EnvironmentFile=-/etc/sysconfig/logstash
ExecStart=/usr/share/logstash/bin/logstash "--path.settings" "/etc/logstash"
Restart=always
WorkingDirectory=/
Nice=19
LimitNOFILE=16384

[Install]
WantedBy=multi-user.target

（当我注释掉上面的用户和群组时，结果相同）

Answer 1

仅仅为了记录，结果问题是systemd没有看到heroku凭据。我问了similar question on AskUbuntu。解决方案是将ubuntu主目录添加到logstash服务，以便它可以访问凭据。通过编辑 / etc / systemd / system 中的<script src="https://ajax.googleapis.com/ajax/libs/jquery/2.1.1/jquery.min.js"></script> <script src="http://intridea.github.io/sketch.js/lib/sketch.js"></script> <canvas id="displaycake_sketch"></canvas> <div> <input type="radio" id="shape1" name="shape_design" value="CIRCLE" onchange="display()"/> O <input type="radio" id="shape2" name="shape_design" value="RECTANGLE" onchange="display()"/> [] </div> <div> <input type="radio" id="color1" name="color_design" value="RED" onchange="display()"/> RED <input type="radio" id="color2" name="color_design" value="BLUE" onchange="display()"/> BLUE </div> <div class="tools"> <a href="#displaycake_sketch" class="btn btn-primary" data-tool="marker"> Marker</a> <a href="#displaycake_sketch" class="btn btn-primary" data-tool="eraser"> Eraser</a> </div>文件并在那里设置logstash.service来完成。

最终logstash.service文件示例：

Environment="Home=/home/ubuntu"

logstash-input-heroku作为服务在ubuntu上运行（logstash 5.2.1）

1 个答案: