Question

我正在尝试将数据聚合器连接到kafka-broker。目的是从HTTP端点收集数据，并将其定期保存到数据库中。两者都使用以下docker-compose.yml在docker中运行：

version: "3"

services:
  zk:
    image: confluentinc/cp-zookeeper:5.0.0
    container_name: zk
    ports:
      - 2181:2181
    environment:
      ZOOKEEPER_CLIENT_PORT: 2181
      ZOOKEEPER_TICK_TIME: 2000
    restart: on-failure:5
  kafka:
    image: confluentinc/cp-kafka:5.0.0
    container_name: kafka
    ports:
      - 9092:9092
      - 9094:9094
    environment:
      KAFKA_ZOOKEEPER_CONNECT: zk:2181
      KAFKA_ADVERTISED_LISTENERS: INSIDE://kafka:9092,OUTSIDE://localhost:9094
      KAFKA_LISTENERS: INSIDE://0.0.0.0:9092,OUTSIDE://0.0.0.0:9094
      KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: INSIDE:PLAINTEXT,OUTSIDE:PLAINTEXT
      KAFKA_INTER_BROKER_LISTENER_NAME: INSIDE
      KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1
      KAFKA_GROUP_INITIAL_REBALANCE_DELAY_MS: 0
      KAFKA_AUTO_CREATE_TOPICS_ENABLE: "true"
      KAFKA_NUM_PARTITIONS: 4
    restart: on-failure:5
  tinygator-metric-aggregator:
    image: tinygator.aggregator:latest
    container_name: tg-aggregator
    environment:
      KAFKA_BOOTSTRAP_SERVER: localhost:9094
      KAFKA_TOPIC: test
      METRIC_ENDPOINT: "http://localhost:15672/api/queues/%2F/Test?sort=message_stats.publish_details.rate&sort_reverse=true&columns=name,message_stats.publish_details.rate,message_stats.deliver_get_details.rate"
      METRIC_METHOD: GET
      METRIC_INTERVAL: 5000
      METRIC_HEADERS: '{"authorization":"Basic Z3Vlc3Q6Z3Vlc3Q="}'
    restart: on-failure:5

Kafka和zk似乎正常启动。但是，聚集器出现以下错误：

tg-aggregator                  | %3|1576348274.043|FAIL|rdkafka#producer-1| [thrd:0.0.0.0:9094/bootstrap]: 0.0.0.0:9094/bootstrap: Connect to ipv4#0.0.0.0:9094 failed: Connection refused (after 4704ms in state CONNECT)
tg-aggregator                  | %3|1576348274.048|ERROR|rdkafka#producer-1| [thrd:0.0.0.0:9094/bootstrap]: 0.0.0.0:9094/bootstrap: Connect to ipv4#0.0.0.0:9094 failed: Connection refused (after 4704ms in state CONNECT)
tg-aggregator                  | %3|1576348274.048|ERROR|rdkafka#producer-1| [thrd:0.0.0.0:9094/bootstrap]: 1/1 brokers are down
tg-aggregator                  | Unhandled exception. System.Net.Http.HttpRequestException: Cannot assign requested address
tg-aggregator                  |  ---> System.Net.Sockets.SocketException (99): Cannot assign requested address
tg-aggregator                  |    at System.Net.Http.ConnectHelper.ConnectAsync(String host, Int32 port, CancellationToken cancellationToken)
tg-aggregator                  |    --- End of inner exception stack trace ---
tg-aggregator                  |    at System.Net.Http.ConnectHelper.ConnectAsync(String host, Int32 port, CancellationToken cancellationToken)
tg-aggregator                  |    at System.Net.Http.HttpConnectionPool.ConnectAsync(HttpRequestMessage request, Boolean allowHttp2, CancellationToken cancellationToken)
tg-aggregator                  |    at System.Net.Http.HttpConnectionPool.CreateHttp11ConnectionAsync(HttpRequestMessage request, CancellationToken cancellationToken)
tg-aggregator                  |    at System.Net.Http.HttpConnectionPool.GetHttpConnectionAsync(HttpRequestMessage request, CancellationToken cancellationToken)
tg-aggregator                  |    at System.Net.Http.HttpConnectionPool.SendWithRetryAsync(HttpRequestMessage request, Boolean doRequestAuth, CancellationToken cancellationToken)
tg-aggregator                  |    at System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
tg-aggregator                  |    at System.Net.Http.HttpClient.FinishSendAsyncBuffered(Task`1 sendTask, HttpRequestMessage request, CancellationTokenSource cts, Boolean disposeCts)
tg-aggregator                  |    at MetricAggregator.Program.Main(String[] args) in /app/Program.cs:line 83
tg-aggregator                  |    at MetricAggregator.Program.<Main>(String[] args)

现在，我和一个同事已经坐了几个小时了（我们都是卡夫卡的新手）。对于在MacOS上编写了聚合器（可以在https://github.com/Azer0s/TinyGator下找到）的家伙来说，不会发生此问题。我在Windows 10上，我的同事在Arch。我们尝试了许多不同的端口号，设置Kafka侦听器和广告侦听器的方法等。总之，我们完全不知所措。

有人有主意吗？

更新：

在阅读了Kafka侦听器之后，我现在修改了docker-compose，更改了聚合器尝试连接的地址：

KAFKA_BOOTSTRAP_SERVER: kafka:9092

但是错误仍然相同。

Answer 1

该错误来自metrics endpoint，在发送卡夫卡消息之前，该错误称为。

这表明您的指标终结点URL不正确，因为localhost将是容器本身（正在拉取指标，本身没有指标或暴露HTTP接口）

该示例看起来像在拉RabbitMQ queue statistics

在任何情况下，kafka:9092在您的上下文中都是正确的

编写聚合器的人不会发生此问题

我敢肯定，如果他给出了无效的指标终结点

无法从其他容器连接到Kafka Docker容器：连接被拒绝-1/1代理已关闭

1 个答案: