Question

我正在开发一个应用程序，该应用程序从sqs队列中读取一条消息，对该数据进行一些处理，然后获取结果并发布到kafka主题。为了在本地测试，我想在我的docker构建中设置一个kafka映像。我目前能够使用docker-compose在本地启动aws-cli，localstack和我的应用程序的容器。另外，我也可以毫无问题地启动kafka和zookeper。我无法使我的应用程序与kafka通信。

我尝试使用两个单独的撰写文件，并且还弄乱了网络。最后，我引用了：https://rmoff.net/2018/08/02/kafka-listeners-explained/。

这是我的docker-compose文件：

version: '3.7'
services:
  localstack:
    image: localstack/localstack:latest
    container_name: localstack
    env_file: .env
    ports:
      # Localstack endpoints for various API. Format is localhost:container
      - '4563-4584:4563-4584'
      - '8080:8080'
    environment:
      - SERVICES=sns:4575,sqs:4576
      - DATA_DIR=/tmp/localstack/data
    volumes:
      # store data locally in 'localstack' folder
      - './localstack:/tmp/localstack'
    networks:
      - my_network

  aws:
    image: mesosphere/aws-cli
    container_name: aws-cli
    # copy local JSON_DATA folder contents into aws-cli container's app folder
    #volumes:
    #  - ./JSON_DATA:/app
    env_file: .env
    # bash entrypoint needed for multiple commands
    entrypoint: /bin/sh -c
    command: >
      " sleep 10;
        aws --endpoint-url=http://localstack:4576 sqs create-queue --queue-name input_queue;
        aws --endpoint-url=http://localstack:4575 sns create-topic --name input_topic;
        aws --endpoint-url=http://localstack:4575 sns subscribe --topic-arn arn:aws:sns:us-east-2:123456789012:example_topic --protocol sqs --notification-endpoint http://localhost:4576/queue/input_queue; "
    networks:
      - my_network
    depends_on:
      - localstack

  my_app:
    build: .
    image: my_app
    container_name: my_app
    env_file: .env
    ports:
      - '9000:9000'
    networks:
      - my_network
    depends_on:
      - localstack
      - aws

  zookeeper:
    image: confluentinc/cp-zookeeper:5.0.0
    container_name: zookeeper
    ports:
      - 2181:2181
    environment:
      ZOOKEEPER_CLIENT_PORT: 2181
    networks:
      - my_network

  kafka:
    image: confluentinc/cp-kafka:5.0.0
    ports:
      - 9092:9092
    depends_on:
      - zookeeper
    environment:
      # For more details see See https://rmoff.net/2018/08/02/kafka-listeners-explained/
      KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181
      KAFKA_ADVERTISED_LISTENERS: INSIDE://localhost:9092
      KAFKA_LISTENERS: INSIDE://localhost:9092
      KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: INSIDE:PLAINTEXT
      KAFKA_INTER_BROKER_LISTENER_NAME: INSIDE
      KAFKA_CREATE_TOPICS: "output_topic:2:2"
    networks:
      - my_network

networks:
  my_network:

我希望不会因为发布到该主题而出现错误。相反，我得到：

kafka: client has run out of available brokers to talk to (Is your cluster reachable?)

任何想法我可能做错了什么？谢谢您的帮助。

Answer 1

通过将侦听器仅设置为本地主机，可以使代理仅在Kafka容器本身（或从主机到容器）中可解析。

如果您希望另一个Docker服务能够访问该容器，则必须将<some protocol>://kafka:<some port>添加到广告侦听器中，并listeners不是本地主机

协议也添加到KAFKA_LISTENER_SECURITY_PROTOCOL_MAP

的位置

FWIW，该博客应涵盖所有这些基础。

如何修复`kafka：客户端用尽了可用的代理进行对话（您的群集是否可以访问？）错误

1 个答案: