我是Kafka的新手,似乎在Apache Kafka的'快速入门'指南中遇到了几个问题: https://kafka.apache.org/quickstart#quickstart_kafkaconnect
最终,我正在尝试学习如何使用许多kafka消息加载kafka队列,因此本快速入门指南的第7步部分似乎相关。
我在这里找到了二进制下载(Scala 2.11 - kafka_2.11-1.1.0.tgz): https://kafka.apache.org/downloads
我最初试图直接跳到第7步,但在找到这个问题后意识到(Kafka Connect implementation errors)我必须在此之前做几步
因此,我顺利完成了第一步:
tar -xzf kafka_2.11-1.1.0.tgz
cd kafka_2.11-1.1.0
然后我按照第2步:
bin/zookeeper-server-start.sh config/zookeeper.properties
但是我收到了错误
ERROR Unexpected exception, exiting abnormally (org.apache.zookeeper.server.ZooKeeperServerMain)
java.net.BindException: Address already in use
at sun.nio.ch.Net.bind0(Native Method)
at sun.nio.ch.Net.bind(Net.java:433)
at sun.nio.ch.Net.bind(Net.java:425)
at sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:223)
at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:74)
at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:67)
at org.apache.zookeeper.server.NIOServerCnxnFactory.configure(NIOServerCnxnFactory.java:90)
at org.apache.zookeeper.server.ZooKeeperServerMain.runFromConfig(ZooKeeperServerMain.java:117)
at org.apache.zookeeper.server.ZooKeeperServerMain.initializeAndRun(ZooKeeperServerMain.java:87)
at org.apache.zookeeper.server.ZooKeeperServerMain.main(ZooKeeperServerMain.java:53)
at org.apache.zookeeper.server.quorum.QuorumPeerMain.initializeAndRun(QuorumPeerMain.java:116)
at org.apache.zookeeper.server.quorum.QuorumPeerMain.main(QuorumPeerMain.java:78)
但是当我在同一步骤中运行下一个命令时:
bin/kafka-server-start.sh config/server.properties
Kafka服务器似乎运行成功了吗?
然后我尝试继续第3步创建主题:
bin/kafka-topics.sh --create --zookeeper localhost:2181 --replication-factor 1 --partitions 1 --topic test
但这会产生错误:
Error while executing topic command : Replication factor: 1 larger than available brokers: 0.
[2018-04-09 14:13:26,908] ERROR org.apache.kafka.common.errors.InvalidReplicationFactorException: Replication factor: 1 larger than available brokers: 0.
(kafka.admin.TopicCommand$)
然后尝试第4步:
bin/kafka-console-producer.sh --broker-list localhost:9092 --topic test
这似乎有用,我可以写一条消息,但后来我得到一个连接错误(这可能是由于前面的步骤没有成功的事实)
kafka_2.11-1.1.0 user$ bin/kafka-console-producer.sh --broker-list localhost:9092 --topic test
>This is a message
[2018-04-09 14:17:52,631] WARN [Producer clientId=console-producer] Connection to node -1 could not be established. Broker may not be available. (org.apache.kafka.clients.NetworkClient)
[2018-04-09 14:17:52,687] WARN [Producer clientId=console-producer] Connection to node -1 could not be established. Broker may not be available. (org.apache.kafka.clients.NetworkClient)
有谁知道为什么会出现这些问题以及如何解决这些问题?我在该教程中找不到关于这些问题的信息
答案 0 :(得分:0)
如错误所示,您在ZK的默认端口上运行了一些东西。关闭它或更改zookeeper属性文件以使用另一个端口。
答案 1 :(得分:0)
地址+-------------+--------------+------------+----------+--------+
|Customer_name|Customer_phone|Customer_age|Order_name|Order_ID|
+-------------+--------------+------------+----------+--------+
|karti |9685684551 |24 |watch |1 |
|raja |8595456552 |22 |cattoy |2 |
+-------------+--------------+------------+----------+--------+
已在使用中。由于Zookeeper无法启动,因此Kafka经纪人也不会开始。 localhost:2181
必须小于或等于可用代理的数量,并且由于没有代理可用,因此将报告以下错误(即使您使用replication-factor
)。
--replication-factor 1
您需要停止在2181中运行的进程,或者将ZK默认端口更改为当前未使用的端口。
要查看端口2181中正在运行的内容(PID),请运行
Error while executing topic command : Replication factor: 1 larger than available brokers: 0.
[2018-04-09 14:13:26,908] ERROR org.apache.kafka.common.errors.InvalidReplicationFactorException: Replication factor: 1 larger than available brokers: 0.
(kafka.admin.TopicCommand$)
如果要杀死该进程,请运行
lsof -i -n -P | grep 2181
其中PID是您可以从kill -9 PID
命令获取的进程ID。
否则,您需要通过修改参数lsof
来更改zookeeper.properties
文件中的端口。最后,您需要相应地更改clientPort=2181
文件中的zookeeper.connect=localhost:2181
参数。