具有动态阈值的PromQL ALERT规则

Question

在Brian Brian这个博客https://www.robustperception.io/using-time-series-as-alert-thresholds之后，我们正尝试将时间序列用作警报阈值

这是记录规则

groups:
  - name: xor-mock
    rules:
      - record: resource:critical
        expr: 0.6
        labels:
          resource_type: cpu:usage
          severity: critical

      - record: resource:critical
        expr: 0.9
        labels:
          resource_type: cpu:usage
          severity: critical
          entity_type: Node

      - record: resource:saturation
        expr: >
          resource > on(resource_type, entity_type) group_left(severity)resource:critical{entity_type !=""}
            or
          resource > on(resource_type) group_left(severity) resource:critical{entity_type =""}

这是失败的promql测试

rule_files:
  - "../prom_support/xor_rule.yml"

evaluation_interval: 1m
tests:
  - interval: 1m
    input_series:
      - series: 'resource{instance="i1", job="node", entity_type="Node", resource_type="cpu:usage", source="node_exporter"}'
        values: '0.8+0x10'
    promql_expr_test:
      - expr: resource:saturation
        eval_time: 1m
        exp_samples:

我们想创建一个promql，其中第一个条件匹配，然后跳过第二个条件，例如if .. else块

我们需要帮助解决promql规则

      resource > on(resource_type, entity_type) group_left(severity)resource:critical{entity_type !=""}
        or
      resource > on(resource_type) group_left(severity) resource:critical{entity_type =""}