Debezium没有为MySQL提供CDC for Embedded版本

Question

我正在使用以下依赖项，

        <dependency>
        <groupId>io.debezium</groupId>
        <artifactId>debezium-connector-oracle</artifactId>
        <version>${version.debezium}</version>
    </dependency>
    <!-- https://mvnrepository.com/artifact/io.debezium/debezium-connector-mysql -->
    <dependency>
        <groupId>io.debezium</groupId>
        <artifactId>debezium-connector-mysql</artifactId>
        <version>${version.debezium}</version>
    </dependency>

<version.debezium>0.8.3.Final</version.debezium>

下面是我的java方法，

public void runMysqlParsser() {

    Configuration config = Configuration.create()
            /* begin engine properties */
            .with("connector.class",
                    "io.debezium.connector.mysql.MySqlConnector")
            .with("offset.storage",
                    "org.apache.kafka.connect.storage.FileOffsetBackingStore")
            .with("offset.storage.file.filename",
                    "/home/mohit/tmp/offset.dat")
            .with("offset.flush.interval.ms", 60000)
            /* begin connector properties */
            .with("name", "my-sql-connector")
            .with("database.hostname", "localhost")
            .with("database.port", 3306)
            .with("database.user", "root")
            .with("database.password", "root")
            .with("server.id", 1)
            .with("database.server.name", "my-app-connector")
            .with("database.history",
                    "io.debezium.relational.history.FileDatabaseHistory")
            .with("database.history.file.filename",
                    "/home/mohit/tmp/dbhistory.dat")
            .with("database.whitelist", "mysql")
            .with("table.whitelist", "mysql.customers")
            .build();
    EmbeddedEngine engine = EmbeddedEngine.create()
            .using(config)
            .notifying(this::handleEvent)
            .build();
    Executor executor = Executors.newSingleThreadExecutor();
    executor.execute(engine);
}

    private void handleEvent(SourceRecord sourceRecord) {
    try {
        LOG.info("Got record :" + sourceRecord.toString());
    } catch (Exception ex) {
        LOG.info("exception in handle event:" + ex);
    }

我的sql配置，

general_log_file = /var/log/mysql/mysql.log
general_log = 1
server-id               = 1
log_bin                 = /var/log/mysql/mysql-bin.log
expire_logs_days        = 10
max_binlog_size   = 100M
binlog_format     = row
binlog_row_image  = full
binlog_rows_query_log_events = on
gtid_mode        =  on
enforce_gtid_consistency   = on

当我运行这段代码时，我正在获取历史日志的偏移量，并且mysql.log文件也正在添加偏移量。但是，当我对表执行任何更新语句时，它没有给我任何日志，即没有调用handleEvent方法。谁能告诉我代码或配置有什么问题吗？

下面是运行Java代码后的日志，

$$ java -jar debezium-gcp-1.0-SNAPSHOT-jar-with-dependencies.jar 

log4j：WARN找不到记录器的附加程序（org.apache.kafka.connect.json.JsonConverterConfig）。 log4j：WARN请正确初始化log4j系统。

log4j：WARN有关更多信息，请参见http://logging.apache.org/log4j/1.2/faq.html#noconfig。 十一月28，2018 1:29:47下午com.debezium.gcp.SampleMysqlEmbededDebezium handleEvent 信息：获得记录：SourceRecord {sourcePartition = {server = my-app-connector}，sourceOffset = {file = mysql-bin.000002，pos = 980，gtids = 31b708c7-ee22-11e8-b8a3-080027fbf50e：1-17， snapshot = true}} ConnectRecord {topic ='my-app-connector'，kafkaPartition = 0，key = Struct {databaseName =}，value = Struct {source = Struct {version = 0.8.3.Final，name = my-app -connector，server_id = 0，ts_sec = 0，file = mysql-bin.000002，pos = 980，row = 0，snapshot = true}，databaseName =，ddl = SET character_set_server = latin1，collat​​ion_server = latin1_swedish_ci;}，时间戳= null，headers = ConnectHeaders（headers =）} 2018年11月28日1:29:47 PM com.github.shyiko.mysql.binlog.BinaryLogClient connect 信息：在31b708c7-ee22-11e8-b8a3-080027fbf50e：1-17（sid：6326，cid：21）连接到localhost：3306

Answer 1

您是否将正确的数据库/表列入白名单？

能否请您观看此演示-https://github.com/debezium/debezium-examples/tree/master/kinesis 只需删除与Kinesis相关的代码，然后仅打印到控制台即可。 还要检查 DROP TABLE IF EXISTS BlockDim; CREATE TABLE BlockDim ( BlockKey bigint NOT NULL AUTO_INCREMENT, BlockID bigint NOT NULL, BlockName varchar(32) NOT NULL, PRIMARY KEY (BlockKey) )ENGINE=InnoDB; DROP TABLE IF EXISTS FacultyDim; CREATE TABLE FacultyDim ( FacultyKey bigint NOT NULL AUTO_INCREMENT, FacultyID bigint NOT NULL, FacultyName varchar(32) NOT NULL, PRIMARY KEY (FacultyKey, FacultyID) )ENGINE=InnoDB; DROP TABLE IF EXISTS `Group`; CREATE TABLE `Group` ( GroupID bigint NOT NULL, FacultyID bigint NOT NULL, GroupName varchar(32) NOT NULL, PRIMARY KEY (GroupID), FOREIGN KEY (FacultyID) REFERENCES FacultyDim(FacultyID) )ENGINE=InnoDB; DROP TABLE IF EXISTS Subblock; CREATE TABLE Subblock ( SubblockKey bigint NOT NULL AUTO_INCREMENT, SubblockID bigint NOT NULL, BlockID bigint NOT NULL, SubblockName varchar(32) NOT NULL, PRIMARY KEY (SubblockKey) )ENGINE=InnoDB; DROP TABLE IF EXISTS Paragraph; CREATE TABLE Paragraph ( ParagraphKey bigint NOT NULL AUTO_INCREMENT, ParagraphID bigint NOT NULL, SubblockID bigint NOT NULL, ParagraphName varchar(64) NOT NULL, PRIMARY KEY (ParagraphKey) )ENGINE=InnoDB; DROP TABLE IF EXISTS RatingDim; CREATE TABLE RatingDim ( RatingKey bigint NOT NULL AUTO_INCREMENT, RatingID bigint NOT NULL, ParagraphID bigint NOT NULL, Score double NOT NULL, StageOfApprove int NOT NULL, Comment varchar(128) NOT NULL, `Date` DATE NOT NULL, PRIMARY KEY (RatingKey) )ENGINE=InnoDB; DROP TABLE IF EXISTS TimeDim; CREATE TABLE TimeDim ( TimeKey bigint NOT NULL AUTO_INCREMENT, `Date` DATE NOT NULL, DayOfWeek varchar(16) NOT NULL, `Month` varchar(16) NOT NULL, `Year` varchar(4) NOT NULL, PRIMARY KEY (`TimeKey`) )ENGINE=InnoDB; DROP TABLE IF EXISTS Rating_Fact; CREATE TABLE Rating_Fact ( TimeKey bigint NOT NULL, BlockKey bigint NOT NULL, FacultyKey bigint NOT NULL, RatingKey bigint NOT NULL, PopularBlock bigint NOT NULL, AvgFacultyScore bigint NOT NULL, StudentsMaxRating bigint NOT NULL, StudentsNotApprovedRating bigint NOT NULL, PRIMARY KEY (TimeKey,BlockKey,FacultyKey,RatingKey), FOREIGN KEY (TimeKey) REFERENCES TimeDim(TimeKey), FOREIGN KEY (BlockKey) REFERENCES BlockDim(BlockKey), FOREIGN KEY (FacultyKey) REFERENCES FacultyDim(FacultyKey), FOREIGN KEY (RatingKey) REFERENCES RatingDim(RatingKey) )ENGINE=InnoDB; 配置选项。恕我直言table.ignore.builtin数据库属于内置数据库，默认情况下会被过滤掉。