如何将高级Kafka配置添加到Landoop`fast-data-dev`映像中

时间:2017-09-28 15:36:31

标签: docker apache-kafka docker-compose

有谁知道如何将自定义Kafka配置选项传递给landoop/fast-data-dev泊坞窗图片?

由于没有办法自定义配置文件和/或配置参数,我到目前为止尝试的是将我自己的server.properties配置文件添加到/opt/confluent/etc/kafka中,方法是将以下内容添加到我的码头撰写文件

landoop:
hostname: 'landoop'
image: 'landoop/fast-data-dev:latest'
expose:
  - '3030'
ports:
  - '3030:3030'
environment:
  - RUNTESTS=0
  - RUN_AS_ROOT=1
volumes:
  - ./docker/landoop/tmp:/tmp
  - ./docker/landoop/opt/confluent/etc/kafka:/opt/confluent/etc/kafka

但是,这会导致Kafka抛出以下日志:

landoop_1       | 2017-09-28 11:53:03,886 INFO exited: broker (exit status 1; not expected)
landoop_1       | 2017-09-28 11:53:04,749 INFO spawned: 'broker' with pid 281
landoop_1       | 2017-09-28 11:53:05,851 INFO success: broker entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
landoop_1       | 2017-09-28 11:53:11,867 INFO exited: rest-proxy (exit status 1; not expected)
landoop_1       | 2017-09-28 11:53:12,604 INFO spawned: 'rest-proxy' with pid 314
landoop_1       | 2017-09-28 11:53:13,024 INFO exited: schema-registry (exit status 1; not expected)
landoop_1       | 2017-09-28 11:53:13,735 INFO spawned: 'schema-registry' with pid 341
landoop_1       | 2017-09-28 11:53:13,739 INFO success: rest-proxy entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)

此外,当我转到http://localhost:3030/kafka-topics-ui/时,我会看到以下内容:

KAFKA REST
/api/kafka-rest-proxy 
CONNECTIVITY ERROR

有什么建议吗?谢谢。

2 个答案:

答案 0 :(得分:0)

我做了几件事,简化了整个过程。这仅适用于开发环境

  1. 以交互式shell作为入口点启动Docker

  2. 在主机网络上启动docker

  3. 对server.properties文件进行必要的更改,包括运行docker的主机IP(如(2)docker在主机网络上运行)

** 4.如果要进行任何高级配置,则可以立即进行**

  1. 运行实际的入口点“ /usr/local/bin/setup-and-run.sh”

使用的实际命令:

  1. 启动容器

sudo docker run -it --entrypoint / bin / bash --net = host --rm -e ADV_HOST = HOSTIP landoop / fast-data-dev:latest

  1. 将以下内容添加到/run/broker/server.properties

advertised.host.name =主机IP

advertised.port = 9092

  1. 运行/usr/local/bin/setup-and-run.sh

答案 1 :(得分:0)


今天,使用最新版本的landoop / fast-data-dev,可以通过将配置选项转换为大写字母,用下划线替换点并以KAFKA_开头来指定自定义Kafka配置选项。 例如,如果要为“ log.retention.bytes”和“ log.retention.hours”设置特定的值,则应在docker compose环境部分中添加以下内容:

environment:
  KAFKA_LOG_RETENTION_BYTES: 1073741824
  KAFKA_LOG_RETENTION_HOURS: 48
  ADV_HOST: 127.0.0.1                  
  RUNTESTS: 0   
  BROWSECONFIGS: 1  

您还可以通过这种方式为其他服务(架构注册表,connect,rest proxy)配置选项。检查文档以获取详细信息https://hub.docker.com/r/landoop/fast-data-dev/

一旦容器启动,您可以通过查看容器内部以下路径中的配置文件来确认这一点:

/run/broker/server.properties

,或者如果已在环境参数上将“ BROWSECONFIGS”设置为1,则也可以通过以下URL通过Landoop UI:

http://127.0.0.1:3030/config/broker/server.properties