Question

Prometheus jmx_exporter项目为Kafka提供了一个官方的示例配置：

https://github.com/prometheus/jmx_exporter/blob/master/example_configs/kafka-0-8-2.yml

Kafka Connect是否有类似的配置将Kafka Connect JMX指标导出到Prometheus？

Answer 1

我正在使用的当前解决方案是官方的Confluent Helm Charts和Docker映像，该映像具有内置的Prometheus度量标准导出：

https://github.com/confluentinc/cp-helm-charts/tree/master/charts/cp-kafka-connect

作为参考，如果您需要非Docker，非Kubernetes解决方案，那么可以100％起作用的先前方法如下：

1）定义文件kafka-connect-jmx.yaml。我将此配置文件存储在/usr/share/中，您可以将其放在其他位置，但是您需要在下面更改路径引用。

whitelistObjectNames: ["kafka.connect:*"]
lowercaseOutputName: false
lowercaseOutputLabelNames: false

rules:
  - pattern: 'status: running'
    value: 1
  - pattern: 'status: *'
    value: 0
  - pattern: ".*"

2）下载https://repo1.maven.org/maven2/io/prometheus/jmx/jmx_prometheus_javaagent/0.12.0/jmx_prometheus_javaagent-0.12.0.jar。我将其放在/usr/share/中，您可以将其放置在其他位置，但是您需要在步骤3中更新路径。

3）运行Kafka Connect connect-distributed时，请确保已设置此环境变量，并且路径与步骤1 + 2中使用的路径匹配

KAFKA_OPTS=-javaagent:/usr/share/jmx_prometheus_javaagent-0.12.0.jar=7072:/usr/share/kafka-connect-jmx.yaml

Answer 2

我一直在寻找相同的东西，并在下面找到。希望能有所帮助：）

https://raw.githubusercontent.com/zenreach/docker-kafka-connect/master/jmx_exporter.yaml

lowercaseOutputName: true
rules:
  - pattern: kafka.connect<type=connect-node-metrics, client-id=(.+), node-id=(.+)><>([a-z-]+)
    name: kafka_connect_node_$3
    labels:
      clientId: "$1"
      nodeId: "$2"
  - pattern: kafka.connect<type=connect-coordinator-metrics, client-id=(.+)><>([a-z-]+)thu
    name: kafka_connect_coordinator_$2
    labels:
      clientId: "$1"
  - pattern: kafka.consumer<type=consumer-node-metrics, client-id=(.+), node-id=(.+)><>([a-z-]+)
    name: kafka_connect_consumer_node_$3
    labels:
      clientId: "$1"
      nodeId: "$2"
  - pattern: kafka.consumer<type=consumer-fetch-manager-metrics, client-id=(.+), topic=(.+)><>([a-z-]+)
    name: kafka_connect_consumer_fetch_manager_$3
    labels:
      clientId: "$1"
      topic: "$2"
  - pattern: kafka.producer<type=producer-topic-metrics, client-id=(.+), topic=(.+)><>([a-z-]+)
    name: kafka_connect_producer_topic_$3
    labels:
      clientId: "$1"
      topic: "$2"
  - pattern: kafka.producer<type=producer-node-metrics, client-id=(.+), node-id=(.+)><>([a-z-]+)
    name: kafka_connect_producer_node_$3
    labels:
      clientId: "$1"
      nodeId: "$2"

Kafka Connect的jmx_prometheus_javaagent配置？

2 个答案: