Kafka JDBC 源连接器和 Oracle DB 错误

时间:2021-04-11 18:23:25

标签: jdbc apache-kafka

虽然我的 Kafka JDBC 连接器适用于一个简单的表,但对于大多数其他表,它失败并显示错误:

<块引用>

Task 抛出一个未捕获且不可恢复的异常 (org.apache.kafka.connect.runtime.WorkerTask:179) org.apache.kafka.connect.errors.ConnectException:错误处理程序中超出容差 在 org.apache.kafka.connect.runtime.errors.RetryWithToleranceOperator.execAndHandleError(RetryWithToleranceOperator.java:178) 在 org.apache.kafka.connect.runtime.errors.RetryWithToleranceOperator.execute(RetryWithToleranceOperator.java:104) 在 org.apache.kafka.connect.runtime.WorkerSourceTask.convertTransformedRecord(WorkerSourceTask.java:290) 在 org.apache.kafka.connect.runtime.WorkerSourceTask.sendRecords(WorkerSourceTask.java:316) 在 org.apache.kafka.connect.runtime.WorkerSourceTask.execute(WorkerSourceTask.java:240) 在 org.apache.kafka.connect.runtime.WorkerTask.doRun(WorkerTask.java:177) 在 org.apache.kafka.connect.runtime.WorkerTask.run(WorkerTask.java:227) 在 java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) 在 java.util.concurrent.FutureTask.run(FutureTask.java:266) 在 java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) 在 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) 在 java.lang.Thread.run(Thread.java:748) 引起:java.lang.IllegalArgumentException:无效的十进制小数位数:127(大于精度:64) 在 org.apache.avro.LogicalTypes$Decimal.validate(LogicalTypes.java:231) 在 org.apache.avro.LogicalType.addToSchema(LogicalType.java:68) 在 org.apache.avro.LogicalTypes$Decimal.addToSchema(LogicalTypes.java:201) 在 io.confluent.connect.avro.AvroData.fromConnectSchema(AvroData.java:943) 在 io.confluent.connect.avro.AvroData.addAvroRecordField(AvroData.java:1058) 在 io.confluent.connect.avro.AvroData.fromConnectSchema(AvroData.java:899) 在 io.confluent.connect.avro.AvroData.fromConnectSchema(AvroData.java:731) 在 io.confluent.connect.avro.AvroData.fromConnectSchema(AvroData.java:725) 在 io.confluent.connect.avro.AvroData.fromConnectData(AvroData.java:364) 在 io.confluent.connect.avro.AvroConverter.fromConnectData(AvroConverter.java:80) 在 org.apache.kafka.connect.storage.Converter.fromConnectData(Converter.java:62) 在 org.apache.kafka.connect.runtime.WorkerSourceTask.lambda$convertTransformedRecord$2(WorkerSourceTask.java:290) 在 org.apache.kafka.connect.runtime.errors.RetryWithToleranceOperator.execAndRetry(RetryWithToleranceOperator.java:128) 在 org.apache.kafka.connect.runtime.errors.RetryWithToleranceOperator.execAndHandleError(RetryWithToleranceOperator.java:162) ……还有 11 个

我正在使用以下命令创建连接器:

<块引用>

curl -X POST http://localhost:8083/connectors -H "Content-Type: application/json" -d '{"name": "jdbc_source_oracle_03","config": {"connector.class": "io.confluent.connect.jdbc.JdbcSourceConnector","connection.url": "jdbc:oracle:thin:@//XOXO:1521/XOXO","connection.user":"XOXO","connection.password" :"XOXO","numeric.mapping":"best_fit","mode":"timestamp","poll.interval.ms":"1000","validate.non.null":"false","table。 whitelist":"POLICY","timestamp.column.name":"CREATED_DATE","topic.prefix":"ora-","transforms": "addTopicSuffix,InsertTopic,InsertSourceDetails,copyFieldToKey,extractValuefromStruct","transforms.InsertTopic .type":"org.apache.kafka.connect.transforms.InsertField$Value","transforms.InsertTopic.topic.field":"messagetopic","transforms.InsertSourceDetails.type":"org.apache.kafka.connect .transforms.InsertField$Value","transforms.InsertSourceDetails.static.field":"messagesource","transforms.InsertSourceDetails.static.value":"JDBC Source Connector from Oracle on asgard","transforms.ad dTopicSuffix.type":"org.apache.kafka.connect.transforms.RegexRouter","transforms.addTopicSuffix.regex":"(.*)","transforms.addTopicSuffix.replacement":"$1-jdbc-02", "transforms.copyFieldToKey.type":"org.apache.kafka.connect.transforms.ValueToKey","transforms.copyFieldToKey.fields":"ID","transforms.extractValuefromStruct.type":"org.apache.kafka.connect .transforms.ExtractField$Key","transforms.extractValuefromStruct.field":"ID"}}'

1 个答案:

答案 0 :(得分:0)

问题与未声明精度和比例的 Number 列有关。 Robin Moffatt 在这里做了很好的解释: https://rmoff.net/2018/05/21/kafka-connect-and-oracle-data-types