Question

docker-compose.yml（https://github.com/wurstmeister/kafka-docker）

version: "2.1"
services:
  zookeeper:
    image: wurstmeister/zookeeper
    ports:
      - "2181:2181"
  kafka:
    image: wurstmeister/kafka
    ports:
      - "9092:9092"
    environment:
      KAFKA_ADVERTISED_HOST_NAME: localhost
      KAFKA_ADVERTISED_PORT: 9092
      KAFKA_CREATE_TOPICS: "test:3:1"
      KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock

尝试在https://kafka.apache.org/quickstart后生成消息时出错：

~/kafka_2.11-1.0.0$ bin/kafka-console-producer.sh --broker-list localhost:9092 --topic test
>gh 
>[2018-01-19 17:28:15,385] ERROR Error when sending message to topic test with key: null, value: 2 bytes with error: (org.apache.kafka.clients.producer.internals.ErrorLoggingCallback)
org.apache.kafka.common.errors.TimeoutException: Expiring 1 record(s) for test-0: 1566 ms has passed since batch creation plus linger time

列出主题：

~/kafka_2.11-1.0.0$ bin/kafka-topics.sh --list --zookeeper localhost:2181
__consumer_offsets
test

为什么呢？感谢

更新

如何设置KAFKA_ADVERTISED_HOST_NAME或网络来制作我的python / java程序或kafka-console-producer.sh（在docker容器外）以通过localhost:9092生成到kafka的消息？

更新

似乎以下docker-compose.yml正常工作

version: "2"

services:
  zookeeper:
    image: "wurstmeister/zookeeper:latest"
    network_mode: "host"
    ports:
      - 2181:2181
  kafkaserver:
    image: "wurstmeister/kafka:latest"
    network_mode: "host"
    ports:
      - 9092:9092
    environment:
      KAFKA_CREATE_TOPICS: "test:3:1"
      KAFKA_ZOOKEEPER_CONNECT: localhost:2181

Answer 1

我有同样的问题。 kafka-docker README中建议的语法与提供的docker-compose.yml不匹配，后者不能正常工作。我终于找到了这篇文章，BEA的更新版本的docker-compose.yml文件对我有用。谢谢！

以下是详细信息。

我正在按照https://bertrandszoghy.wordpress.com/2018/05/03/building-the-hyperledger-fabric-vm-and-docker-images-version-1-1-from-scratch/所述设置的Ubuntu 16.04虚拟映像上运行wurstmeister / kafka-docker

我的docker-compose.yml文件：

version: '2'
services:
  zookeeper:
    image: "wurstmeister/zookeeper:latest"
    network_mode: "host"
    ports:
      - "2181:2181"
  kafka:
    image: "wurstmeister/kafka:latest"
    network_mode: "host"
    ports:
      - 9092:9092
    environment:
      KAFKA_LISTENERS: PLAINTEXT://:9092
      KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://172.17.0.1:9092
      KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181
      KAFKA_CREATE_TOPICS: "BertTopic:3:1"

在同一个VM上我安装了NodeJs：

curl -sL https://deb.nodesource.com/setup_6.x | sudo -E bash –
sudo apt-get install -y nodejs
cd
mkdir nodecode
cd nodecode
sudo npm install -g node-pre-gyp
sudo npm install kafka-node

然后我运行了以下程序来产生一些消息：

var kafka = require('kafka-node'),
    Producer = kafka.Producer,
    KeyedMessage = kafka.KeyedMessage,
    client = new kafka.Client(),
    producer = new Producer(client),
    km = new KeyedMessage('key', 'message'),
    payloads = [
        { topic: 'BertTopic', messages: 'first test message', partition: 0 },
        { topic: 'BertTopic', messages: 'second test message', partition: 0 }
    ];
producer.on('ready', function () {
    producer.send(payloads, function (err, data) {
        console.log(data);
        process.exit(0);
    });
});

producer.on('error', function (err) {
console.log('ERROR: ' + err.toString());
});

返回了：

{ BertTopic: { '0': 0 } }

我运行第二个NodeJs程序来使用（最后）消息：

var options = {
    fromOffset: 'latest'
};

var kafka = require('kafka-node'),
    Consumer = kafka.Consumer,
    client = new kafka.Client(),
    consumer = new Consumer(
        client,
        [
            { topic: 'BertTopic', partition: 0 }
        ],
        [
                {
                        autoCommit: false
                },
                options =
                {
                        fromOffset: 'latest'
                }
        ]
    );

返回了：

{ topic: 'BertTopic',
  value: 'first test message',
  offset: 0,
  partition: 0,
  highWaterOffset: 2,
  key: null }
{ topic: 'BertTopic',
  value: 'second test message',
  offset: 1,
  partition: 0,
  highWaterOffset: 2,
  key: null }

我还有第三个NodeJs程序，用于显示我的博文https://bertrandszoghy.wordpress.com/2017/06/27/nodejs-querying-messages-in-apache-kafka/中列出的主题中的所有历史消息

希望这可以帮助别人。

kafka：无法在docker内部向kafka发送消息

1 个答案: