Question

我正在运行debezium来从mysql bin日志中捕获数据，并使用kafka-connect在Kafka上发布。我今天收到此错误：

[2019-06-18 11:32:38,175] ERROR Failed to properly convert data value for 'wallet_db_v3.user_registrations.msisdn' of type VARCHAR for row [824, 2, 2019-06-18T06:20:26]: (io.debezium.relational.TableSchemaBuilder:253)
java.lang.IllegalArgumentException: Unexpected value for JDBC type 12 and column msisdn VARCHAR(50) CHARSET utf8 NOT NULL: class=class java.lang.Integer
at io.debezium.jdbc.JdbcValueConverters.handleUnknownData(JdbcValueConverters.java:1164)
at io.debezium.jdbc.JdbcValueConverters.convertValue(JdbcValueConverters.java:1207)
at io.debezium.connector.mysql.MySqlValueConverters.convertString(MySqlValueConverters.java:380)
at io.debezium.connector.mysql.MySqlValueConverters.lambda$converter$13(MySqlValueConverters.java:293)
at io.debezium.relational.TableSchemaBuilder.lambda$createValueGenerator$2(TableSchemaBuilder.java:249)
at io.debezium.relational.TableSchema.valueFromColumnData(TableSchema.java:145)
at io.debezium.connector.mysql.RecordMakers$1.insert(RecordMakers.java:244)
at io.debezium.connector.mysql.RecordMakers$RecordsForTable.create(RecordMakers.java:456)
at io.debezium.connector.mysql.BinlogReader.handleInsert(BinlogReader.java:833)
at io.debezium.connector.mysql.BinlogReader.handleEvent(BinlogReader.java:501)
at com.github.shyiko.mysql.binlog.BinaryLogClient.notifyEventListeners(BinaryLogClient.java:1095)
at com.github.shyiko.mysql.binlog.BinaryLogClient.listenForEventPackets(BinaryLogClient.java:943)
at com.github.shyiko.mysql.binlog.BinaryLogClient.connect(BinaryLogClient.java:580)
at com.github.shyiko.mysql.binlog.BinaryLogClient$7.run(BinaryLogClient.java:825)
at java.lang.Thread.run(Thread.java:748)

表结构：

'id', 'int(10) unsigned', 'NO', 'PRI', NULL, 'auto_increment'
'msisdn', 'varchar(50)', 'NO', '', NULL, ''
'device_id', 'varchar(50)', 'NO', '', NULL, ''
'status', 'enum(\'waiting_for_validation\',\'approved\',\'rejected\',\'expired\')', 'NO', '', NULL, ''
'rejection_reason', 'enum(\'max_otp_validations_reached\',\'canceled_by_user\')', 'YES', '', NULL, ''
'updated_at', 'datetime', 'YES', '', 'CURRENT_TIMESTAMP', 'on update CURRENT_TIMESTAMP'
'created_at', 'datetime', 'NO', '', NULL, ''
'channel', 'varchar(50)', 'YES', '', NULL, ''

整行应如下所示：

select * from user_registrations where id = 824;
['824', '+115004937263', '56DA9A2FD4D91320D01514ED497C719E5DC9323F', 'approved', NULL, '2019-06-18 06:20:26', '2019-06-18 06:20:23', NULL]

如果现在查询表，则此行显示为完整。

我怀疑debezium连接器在将记录完全写入mysql binlog中之前正在读取记录。是否有人配置debezium的方式可以确保在将记录完全写入Binlog后读取它

供参考的my.cnf设置：

log-bin=/var/log/mysql/mysql-bin
binlog_format=row
binlog_row_image=full

Answer 1

问题出在源表上-它是 NDB 表，无法通过将binlog_row_image参数设置为full来控制。

配置debezium连接器以读取mysql二进制日志而没有任何部分数据

1 个答案: