我遇到了一个奇怪的问题,我的镜像创建者能够在目标集群中镜像主题名称,但是当我尝试从中使用消息时看不到任何消息。下面是我的配置文件–
Kafka version : kafka_10.0.1
All Certs /Keys are my organization issued certificate so no Error for SSL
Source Cluster :
==============
Zookeeeper Properties
dataDir=/home/kafka/zookeeper
clientPort=2181
maxClientCnxns=3
Server Properties
broker.id=1
listeners= SSL://123.456.789.110:9092
advertised.listeners=SSL://123.456.789.110:9092
security.inter.broker.protocol=SSL
num.network.threads=3
num.io.threads=8
auto.create.topics.enable=true
ssl.client.auth=none
ssl.keystore.location=/etc/keys/keystore
ssl.keystore.password=test1234
ssl.truststore.location=/etc/CApath/trustedstore
ssl.truststore.password=test1234
ssl.enabled.protocols=TLSv1.2
log.dirs=/home/kafka/kafka-logs
zookeeper.connect=123.456.789.110:2181
Kafka Up and Running . Log persistently logging in Source Cluster
Target Cluster :
================
Zookeeeper Properties
dataDir=/home/kafka/zookeeper
clientPort=2181
maxClientCnxns=3
Server Properties
broker.id=1
listeners= SSL://234.456.789.110:9092
advertised.listeners=SSL://sl73easapd04.visa.com:9092
security.inter.broker.protocol=SSL
num.network.threads=3
num.io.threads=8
ssl.client.auth=none
ssl.keystore.location=/etc/keys/keystore
ssl.keystore.password=test1234
ssl.truststore.location=/etc/CApath/trustedstore
ssl.truststore.password=test1234
ssl.enabled.protocols=TLSv1.2
ssl.keystore.type = JKS
ssl.truststore.type = JKS
log.dirs=/data/visingh/kafka/kafka-logs
num.partitions=1
num.recovery.threads.per.data.dir=1
log.retention.hours=168
log.segment.bytes=1073741824
log.retention.check.interval.ms=300000
zookeeper.connect=234.456.789.110:2181
zookeeper.connection.timeout.ms=6000
mm_consumer.properties :
bootstrap.servers=123.456.789.110:9092
exclude.internal.topics = true
auto.offset.reset=earliest
security.protocol = SSL
ssl.truststore.location=/etc/CApath/trustedstore
ssl.truststore.password=Test1234
ssl.enabled.protocols= TLSv1.2
ssl.keystore.location=/etc/keys/keystore
ssl.keystore.password=Test1234
ssl.truststore.type=JKS
ssl.keystore.type=JKS
zookeeper.connection.timeout.ms=6000
group.id=test-consumer-group
client.id=mirror_maker_consumer
consumer.timeout.ms=50000
mm_producer.properties :
bootstrap.servers=234.456.789.110:9092
acks=1
batch.size=100
security.protocol = SSL
ssl.truststore.location=/etc/CApath/trustedstore
ssl.truststore.password=Test1234
ssl.enabled.protocols=TLSv1.2
ssl.keystore.location=/etc/keys/keystore
ssl.keystore.password=Test1234
ssl.truststore.type=JKS
ssl.keystore.type=JKS
compression.type=gzip
然后我使用以下命令启动Mirror-Maker,该命令运行良好,并且主题Target-logging在目标集群中创建,但未在目标集群上登录。
bin / kafka-run-class.sh kafka.tools.MirrorMaker --new.consumer --consumer.config config / mm_consumer.properties --producer.config config / mm_producer.properties --whitelist *
kafka MirroMaker已启动,但目标群集上没有日志。
我执行了另一个使用主题名称和以下命令的镜像制造商命令,该命令在目标hostCluster上失败
/bin/kafka-mirror-maker.sh --new.consumer --consumer.config config / mm_consumer.properties --producer.config config / mm_producer.properties --whitelist应用日志
[2018-06-22 03:59:27,536] FATAL [mirrormaker-thread-0] Mirror maker thread
failure due to (kafka.tools.MirrorMaker$MirrorMakerThread)
java.lang.IllegalArgumentException: Invalid timestamp -1
at org.apache.kafka.clients.producer.ProducerRecord.<init> (ProducerRecord.java:60)
at kafka.tools.MirrorMaker$defaultMirrorMakerMessageHandler$.handle(MirrorMaker.scala:678)
at kafka.tools.MirrorMaker$MirrorMakerThread.run(MirrorMaker.scala:414)
[2018-06-22 03:59:27,563] FATAL [mirrormaker-thread-0] Mirror maker thread exited abnormally, stopping the whole mirror maker. (kafka.tools.MirrorMaker$MirrorMakerThread)
还要运行包含引号的“。*”,而不只是*。
bin / kafka-run-class.sh kafka.tools.MirrorMaker --new.consumer --consumer.config config / mm_consumer.properties --producer.config config / mm_producer.properties --whitelist“。*” < / p>
[2018-06-22 04:22:01,647] FATAL [mirrormaker-thread-0] Mirror maker thread failure due to (kafka.tools.MirrorMaker$MirrorMakerThread)
java.lang.IllegalArgumentException: Invalid timestamp -1
at org.apache.kafka.clients.producer.ProducerRecord.<init>(ProducerRecord.java:60)
at kafka.tools.MirrorMaker$defaultMirrorMakerMessageHandler$.handle(MirrorMaker.scala:678)
at kafka.tools.MirrorMaker$MirrorMakerThread.run(MirrorMaker.scala:414)
[2018-06-22 04:22:01,674] FATAL [mirrormaker-thread-0] Mirror maker thread exited abnormally, stopping the whole mirror maker. (kafka.tools.MirrorMaker$MirrorMakerThread)
我想念什么吗?