当控制器成为第二个节点(id 2)时,zookeeper和Kafka无法正常工作

时间:2019-07-07 07:02:27

标签: apache-kafka cluster-computing apache-zookeeper kafka-cluster

我们有一个带有三个节点的Kafka集群。我们有一个奇怪的问题。当控制器成为第二个节点(id 2)时,zookeeper和Kafka无法正常工作。

zookeeper配置:

#static code if header at 0,0 position
import xlrd 

loc = ("path of file") 

wb = xlrd.open_workbook(loc) 
sheet = wb.sheet_by_index(0) 

# For row 0 and column 0 
sheet.cell_value(0, 0) 

for i in range(sheet.ncols): 
    print(sheet.cell_value(0, i)) 

kafka配置:

maxClientCnxns=50
tickTime=2000
initLimit=10
syncLimit=5
dataDir=/var/lib/zookeeper
clientPort=2181
server.1=x.x.x.x:2888:3888
server.2=x.x.x.x:2888:3888
server.3=x.x.x.x:2888:3888

卡夫卡错误:

broker.id=2
delete.topic.enable=true
advertised.listeners=PLAINTEXT://x.x.x.x:9092
num.network.threads=16
num.io.threads=16
socket.send.buffer.bytes=102400
socket.receive.buffer.bytes=102400
socket.request.max.bytes=104857600
log.dirs=/kafka-logs
num.partitions=1
num.recovery.threads.per.data.dir=1
offsets.topic.replication.factor=1
transaction.state.log.replication.factor=1
transaction.state.log.min.isr=1
log.retention.hours=48
log.segment.bytes=1073741824
log.retention.check.interval.ms=300000
zookeeper.connect=x.x.x.x:2181,x.x.x.x:2181,x.x.x.x:2181/kafka
zookeeper.connection.timeout.ms=6000
group.initial.rebalance.delay.ms=0
auto.create.topics.enable=false
leader.imbalance.check.interval.seconds=60
auto.leader.rebalance.enable=false

和动物园管理员错误:

[2019-07-07 09:28:30,201] INFO [Partition partition-name-11 broker=1] Shrinking ISR from 1,2 to 1 (kafka.cluster.Partition)
[2019-07-07 09:28:30,202] INFO [Partition partition-name-11 broker=1] Cached zkVersion [1208] not equal to that in zookeeper, skip updating ISR (kafka.cluster.Partition)

[2019-07-07 09:28:30,328] INFO [ReplicaFetcher replicaId=1, leaderId=3, fetcherId=0] Remote broker is not the leader for partition partition-name-4, which could indicate that the partition is being moved (kafka.server.ReplicaFetcherThread)


[2019-07-07 09:28:03,088] WARN [LeaderEpochCache partition-name-8] New epoch entry EpochEntry(epoch=1670, startOffset=0) caused truncation of conflicting entries ListBuffer(EpochEntry(epoch=1668, startOffset=0)). Cache now contains 1 entries. (kafka.server.epoch.LeaderEpochFileCache)
[2019-07-07 09:28:03,088] INFO [Partition partition-name-11 broker=1] Cached zkVersion [2920] not equal to that in zookeeper, skip updating ISR (kafka.cluster.Partition)

[2019-07-07 10:21:59,386] INFO [Partition partition-name-4 broker=2] partition-name-4 starts at Leader Epoch 1105 from offset 0. Previous Leader Epoch was: 1104 (kafka.cluster.Partition)

0 个答案:

没有答案