错误无效的配置,在Kafka集群中异常退出

时间:2019-01-16 14:05:36

标签: docker apache-kafka apache-zookeeper

我有三个不同的节点,每个节点上都有带有Ubuntu的docker。我可以在本地docker上运行zookeeper服务器(我的意思是没有对集群进行任何配置),但是我想使kafka集群具有这三个节点;实际上,我在每个节点上安装了docker,并在其上加载了Ubuntu。我在docker环境中将"zookeeper.properties"配置为“ 150.20.11.157”,如下所示:

dataDir=/tmp/zookeeper/data
tickTime=2000
initLimit=10
syncLimit=5
server.1=0.0.0.0:2888:3888
server.2=150.20.11.134:2888:3888
server.3=150.20.11.137:2888:3888
clientPort=2186

对于节点150.20.11.134,docker环境中的“ zookeeper.properties”文件如下所示:

dataDir=/tmp/zookeeper/data
tickTime=2000
initLimit=10
syncLimit=5
server.1=150.20.11.157:2888:3888
server.2=0.0.0.0:2888:3888
server.3=150.20.11.137:2888:3888
clientPort=2186

对于节点150.20.11.137,泊坞窗环境中的“ zookeeper.properties”文件如下:

dataDir=/tmp/zookeeper/data
tickTime=2000
initLimit=10
syncLimit=5
server.1=150.20.11.157:2888:3888
server.2=150.20.11.134:2888:3888
server.3=0.0.0.0:2888:3888
clientPort=2186

此外,我为节点150.20.11.157设置“ server.properties”:

broker.id=0
port=9092
listeners = PLAINTEXT://150.20.11.157:9092
log.dirs=/tmp/kafka-logs 
zookeeper.connect=150.20.11.157:2186,150.20.11.134:2186,
150.20.11.137:2186
节点150.20.11.134的

“ server.properties”是:

broker.id=1
port=9092
listeners = PLAINTEXT://150.20.11.134:9092
log.dirs=/tmp/kafka-logs 
zookeeper.connect=150.20.11.157:2186,150.20.11.134:2186,
150.20.11.137:2186
节点150.20.11.137的

“ server.properties”是:

broker.id=2
port=9092
listeners = PLAINTEXT://150.20.11.137:9092
log.dirs=/tmp/kafka-logs 
zookeeper.connect=150.20.11.157:2186,150.20.11.134:2186,
150.20.11.137:2186

问题是当我在每个节点的docker上运行zookeeper服务器时。我收到此错误:

[2019-01-16 12:45:54,588] INFO Reading configuration from: ./config/zookeeper.properties 
(org.apache.zookeeper.server.quorum.QuorumPeerConfig)

[2019-01-16 12:45:54,601] INFO Resolved hostname: 172.28.10.137 to address: /172.28.10.137 (org.apache.zookeeper.server.quorum.QuorumPeer)

[2019-01-16 12:45:54,603] INFO Resolved hostname: 0.0.0.0 to address: /0.0.0.0 (org.apache.zookeeper.server.quorum.QuorumPeer)

[2019-01-16 12:45:54,603] INFO Resolved hostname: 172.28.10.157 to address: /172.28.10.157 (org.apache.zookeeper.server.quorum.QuorumPeer)

[2019-01-16 12:45:54,603] INFO Defaulting to majority quorums (org.apache.zookeeper.server.quorum.QuorumPeerConfig)

[2019-01-16 12:45:54,604] ERROR Invalid config, exiting abnormally (org.apache.zookeeper.server.quorum.QuorumPeerMain)

org.apache.zookeeper.server.quorum.QuorumPeerConfig$ConfigException: Error processing ./config/zookeeper.properties
    at org.apache.zookeeper.server.quorum.QuorumPeerConfig.parse(QuorumPeerConfig.java:156)
    at org.apache.zookeeper.server.quorum.QuorumPeerMain.initializeAndRun(QuorumPeerMain.java:104)
    at org.apache.zookeeper.server.quorum.QuorumPeerMain.main(QuorumPeerMain.java:81)
Caused by: java.lang.IllegalArgumentException: /tmp/zookeeper/data/myid file is missing
    at org.apache.zookeeper.server.quorum.QuorumPeerConfig.parseProperties(QuorumPeerConfig.java:408)
    at org.apache.zookeeper.server.quorum.QuorumPeerConfig.parse(QuorumPeerConfig.java:152)    
... 2 more

Invalid config, exiting abnormally

请告诉我如何创建一个具有三个docker的Kafka集群,每个docker都在一个物理节点上?

谢谢。

1 个答案:

答案 0 :(得分:2)

我也遇到了这个问题。

我认为这一点的线索在日志中,即

  

原因:java.lang.IllegalArgumentException:    / tmp / zookeeper / data / myid文件丢失   org.apache.zookeeper.server.quorum.QuorumPeerConfig.parseProperties(QuorumPeerConfig.java:408)

if (window.location.origin === "red.abc.com") { var link = document.createElement('link'); link.rel = 'stylesheet'; link.href = window.location.origin + 'red.css'; document.head.appendChild(link); } 文件中的以下各行提供了Zookeeper集成的详细信息:

zookeeper.properties

其中一个zookeeper服务器启动时,它通过查看其自己的数据目录中的server.1=0.0.0.0:2888:3888 server.2=150.20.11.134:2888:3888 server.3=150.20.11.137:2888:3888 文件(在本例中为myid)来知道它是哪台服务器。

因此,您要做的就是在每个服务器的上述目录中创建一个名称为/tmp/zookeeper/data的文件,并只写x = {myid1或{{ 1}}(它们对应于server.zookeeper.properties文件中的2。)

参考链接-Apache Zookeeper

希望这会有所帮助!