Question

我正在使用Debezium作为CDC工具从MySql流数据。在将Debezium MySQL连接器安装到Confluent OSS集群后，我试图将MySQL bin_log更改捕获到Kafka主题中。创建连接器时，在拍摄数据库快照后，我会遇到一系列连续的错误。

我检查了MySql bin_log是否为ON，并尝试使用不同的序列化程序重新启动架构注册表和连接器。但是我遇到了同样的错误。

错误日志显示：

[2019-06-21 13:56:14,885] INFO Step 8: - Completed scanning a total of 955 rows from table 'mydb.test' after 00:00:00.086 (io.debezium.connector.mysql.SnapshotReader:565)
[2019-06-21 13:56:14,886] INFO Step 8: scanned 1758 rows in 2 tables in 00:00:00.383 (io.debezium.connector.mysql.SnapshotReader:601)
[2019-06-21 13:56:14,886] INFO Step 9: committing transaction (io.debezium.connector.mysql.SnapshotReader:635)
[2019-06-21 13:56:14,887] INFO Completed snapshot in 00:00:01.055 (io.debezium.connector.mysql.SnapshotReader:701)
[2019-06-21 13:56:14,965] WARN [Producer clientId=producer-5] Error while fetching metadata with correlation id 11 : {kbserver=UNKNOWN_TOPIC_OR_PARTITION} (org.apache.kafka.clients.NetworkClient:968)
[2019-06-21 13:56:15,066] WARN [Producer clientId=producer-5] Error while fetching metadata with correlation id 12 : {kbserver=UNKNOWN_TOPIC_OR_PARTITION} (org.apache.kafka.clients.NetworkClient:968)
[2019-06-21 13:56:15,168] WARN [Producer clientId=producer-5] Error while fetching metadata with correlation id 13 : {kbserver=UNKNOWN_TOPIC_OR_PARTITION} (org.apache.kafka.clients.NetworkClient:968)
[2019-06-21 13:56:15,269] WARN [Producer clientId=producer-5] Error while fetching metadata with correlation id 14 : {kbserver=UNKNOWN_TOPIC_OR_PARTITION} (org.apache.kafka.clients.NetworkClient:968)
[2019-06-21 13:56:15,370] WARNDebezium [Producer clientId=producer-5] Error while fetching metadata with correlation id 15 : {kbserver=UNKNOWN_TOPIC_OR_PARTITION}

我要发送的连接器有效载荷如下：

{
      "name": "debezium-connector",
      "config": {
            "connector.class": "io.debezium.connector.mysql.MySqlConnector",
            "tasks.max": "1",
            "key.serializer": "io.confluent.connect.avro.AvroConverter",
            "value.serializer": "io.confluent.connect.avro.AvroConverter",
            "database.hostname": "localhost",
            "database.port": "3306",
            "database.user": "test",
            "database.password": "test@123",
            "database.whitelist": "mydb",
            "table.whitelist": "mydb.test",
            "database.server.id": "1",
            "database.server.name": "kbserver",
            "database.history.kafka.bootstrap.servers": "kafka:9092",
            "database.history.kafka.topic": "db-schema.mydb",
            "include.schema.changes": "true"
       }
    }

有人知道为什么会发生这种情况，或者我该如何解决？

Answer 1

请参考database.whitelist和table.whitelist。它们不一致。

根据数据库名称，它应该是mydb和mydb.test或db和db.test。

Kafka生产者创建Debezium MySQL连接器后无法创建主题并引发连续错误

1 个答案: