通知警报:超出了上下文期限(可能出现代理问题)

时间:2019-04-14 18:41:05

标签: amazon-web-services amazon-ec2 proxy prometheus-alertmanager

我正在尝试将EC2 AWS中ubuntu上的Alermanager服务器连接到Slack,但是我收到了以下错误:

Apr 14 18:22:26 prometheus-db-v01-01a.myserver.com alertmanager[5854]: level=error ts=2019-04-14T18:22:26.658601495Z caller=dispatch.go:177 component=dispatcher msg="Notify for alerts failed" num_alerts=1 err="Post <redacted>: context deadline exceeded"

我的alertmanager.yum看起来像:

templates:
- '/etc/alertmanager/template/slack.tmpl'

route:
  receiver: slack_general
  repeat_interval: 5m
  group_by: [alertname]
  routes:
# severity=info alerts will not try to match to any other rule
    - match:
        severity: info
      receiver: slack_general

receivers:
- name: slack_general
  slack_configs:
  - api_url: https://hooks.slack.com/services/ID
    send_resolved: true
    username: 'Prometheus-bot'
    channel: '#errors'
    title: '{{ template "slack.my.title" . }}'
    text: '{{ template "slack.my.text" . }}'

我可以看到以下警报触发我的测试警报:

root@prometheus-db-v01-01a:~# amtool alert --alertmanager.url=http://localhost:9093 -v
Alertname         Starts At                Summary
Cassandra_yellow  2019-04-14 18:11:56 UTC  The cassandra  cluster is in yellow state

我从systemd开始了普罗米修斯:

root@prometheus-db-v01-01a:~# cat /etc/systemd/system/alertmanager.service
[Unit]
Description=Prometheus Alertmanager Service
Wants=network-online.target
After=network.target

[Service]
Environment=https_proxy=http://proxy:80/
Environment=http_proxy=http://proxy:80/
User=alertmanager
Group=alertmanager
Type=simple
ExecStart=/usr/local/bin/alertmanager \
    --config.file /etc/alertmanager/alertmanager.yml \
    --storage.path /var/lib/alertmanager/data
Restart=always

[Install]
WantedBy=multi-user.target
root@prometheus-db-v01-01a:~#

我尝试使用2 Environment变量设置代理,但结果相同

服务器使用代理与Slack进行通信,我可以通过以下方式对其进行测试:

root@prometheus-db-v01-01a:~# curl -X POST --data-urlencode "payload={\"channel\": \"#errors\", \"username\": \"webhookbot\", \"text\": \"This is posted to #errors and comes from a bot named webhookbot.\", \"icon_emoji\": \":ghost:\"}" https://hooks.slack.com/services/ID

有效,而

root@prometheus-db-v01-01a:~# curl -X POST --noproxy "*" --data-urlencode "payload={\"channel\": \"#errors\", \"username\": \"webhookbot\", \"text\": \"This is posted to #errors and comes from a bot named webhookbot.\", \"icon_emoji\": \":ghost:\"}" https://hooks.slack.com/services/ID
curl: (7) Failed to connect to hooks.slack.com port 443: Connection timed out

以上所有命令均来自alertmanger服务器。

我是否认为这是代理问题?我该如何设置?

预先感谢

1 个答案:

答案 0 :(得分:0)

问题已解决:它是代理服务器,需要在alertmanager.yml文件中设置

templates:
- '/etc/alertmanager/template/slack.tmpl'

global:
  http_config:
    proxy_url: 'http://proxy:80/'

route:
  receiver: slack_general
  repeat_interval: 5m
  group_by: [alertname]
  routes:

    - match:
        severity: minor
      receiver: slack_general

receivers:
- name: slack_general
  slack_configs:
[...]