Question

我正在使用5节点kafka和5节点zookeeper集群。 Zookeeper工作正常。卡夫卡一直工作，直到我在卡夫卡创建一个主题。所以，步骤是：

java -jar -Xmx4G -Xms256M /home/desktop/Documents/Run/New_Version/wine.jar batch  - Stop when parse started
java -jar -Xmx4G -Xms256M /home/desktop/Documents/Run/New_Version/wine.jar parse - Successfuly run the parse function

我尝试使用kafka-0.8.1.1和kafka-0.8.2.2，但我得到了同样的错误。我正在使用以下配置文件：

1. Setup Zookeeper cluster and start all zookeeper nodes
2. Setup Kafka cluster and start all kafka nodes
3. Everything is fine upto here.
4. Create a topic.
5. Following error starts to pop up continuously and squeezes all the disk space.

[2015-11-11 10:00:01,289] INFO Reconnect due to socket error: java.nio.channels.ClosedChannelException (kafka.consumer.SimpleConsumer)
[2015-11-11 10:00:01,289] WARN [ReplicaFetcherThread-1-1], Error in fetch Name: FetchRequest; Version: 0; CorrelationId: 287237122; ClientId: ReplicaFetcherThread-1-1; ReplicaId: 4; MaxWait: 500 ms; MinBytes: 1 bytes; RequestInfo: [DataPLFLogger,3] -> PartitionFetchInfo(0,1048576). Possible cause: java.nio.channels.ClosedChannelException (kafka.server.ReplicaFetcherThread)
[2015-11-11 10:00:01,413] INFO Reconnect due to socket error: java.nio.channels.ClosedChannelException (kafka.consumer.SimpleConsumer)
[2015-11-11 10:00:01,413] WARN [ReplicaFetcherThread-1-1], Error in fetch Name: FetchRequest; Version: 0; CorrelationId: 287237123; ClientId: ReplicaFetcherThread-1-1; ReplicaId: 4; MaxWait: 500 ms; MinBytes: 1 bytes; RequestInfo: [DataPLFLogger,3] -> PartitionFetchInfo(0,1048576). Possible cause: java.nio.channels.ClosedChannelException (kafka.server.ReplicaFetcherThread)
[2015-11-11 10:00:01,413] INFO Reconnect due to socket error: java.nio.channels.ClosedChannelException (kafka.consumer.SimpleConsumer)
[2015-11-11 10:00:01,413] WARN [ReplicaFetcherThread-1-1], Error in fetch Name: FetchRequest; Version: 0; CorrelationId: 287237124; ClientId: ReplicaFetcherThread-1-1; ReplicaId: 4; MaxWait: 500 ms; MinBytes: 1 bytes; RequestInfo: [DataPLFLogger,3] -> PartitionFetchInfo(0,1048576). Possible cause: java.nio.channels.ClosedChannelException (kafka.server.ReplicaFetcherThread)
[2015-11-11 10:00:01,413] INFO Reconnect due to socket error: java.nio.channels.ClosedChannelException (kafka.consumer.SimpleConsumer)
[2015-11-11 10:00:01,413] WARN [ReplicaFetcherThread-1-1], Error in fetch Name: FetchRequest; Version: 0; CorrelationId: 287237125; ClientId: ReplicaFetcherThread-1-1; ReplicaId: 4; MaxWait: 500 ms; MinBytes: 1 bytes; RequestInfo: [DataPLFLogger,3] -> PartitionFetchInfo(0,1048576). Possible cause: java.nio.channels.ClosedChannelException (kafka.server.ReplicaFetcherThread)
[2015-11-11 10:00:01,413] INFO Reconnect due to socket error: java.nio.channels.ClosedChannelException (kafka.consumer.SimpleConsumer)
[2015-11-11 10:00:01,414] WARN [ReplicaFetcherThread-1-1], Error in fetch Name: FetchRequest; Version: 0; CorrelationId: 287237126; ClientId: ReplicaFetcherThread-1-1; ReplicaId: 4; MaxWait: 500 ms; MinBytes: 1 bytes; RequestInfo: [DataPLFLogger,3] -> PartitionFetchInfo(0,1048576). Possible cause: java.nio.channels.ClosedChannelException (kafka.server.ReplicaFetcherThread)

我还尝试从一个代理telnet到其他kafka代理，并尝试telnet zookeeper节点。每个连接都成功。

请帮助我解决这个问题。

Answer 1

创建Kafka主题

<强>语法：

kafka-topics.sh --zookeeper <$ZOOKEEPER_HOST:2181> --create --topic <topic-name> --partitions <no of partitions> --replication-factor <no of replicas>

示例：

kafka-topics.sh --zookeeper node1:2181 --create --topic squid --partitions 1 --replication-factor 1

卡夫卡失败并挤压整箱磁盘空间

1 个答案: