debezium mysql 连接器因 Java 堆空间问题而失败

时间:2021-04-19 15:56:40

标签: java mysql kubernetes debezium

我们在 GKE 集群上部署了 debezium mysql connectors。但是 pod 失败,原因是: java.lang.OutOfMemoryError: Java heap space 。将 jvm 选项增加到 10g,但仍然因同样的问题而失败。

以下是我的配置属性:

singleTopicMode=true 
databaseManagementSystem=mysql
debezium.offset.flush.timeout.ms=60000
debezium.offset.flush.interval.ms=10000
debezium.max.request.size=1048
debezium.max.queue.size=65536
debezium.max.batch.size=16384
debezium.HEAP_OPTS='-Xms10g -Xmx10g'                                                           

完整日志:

org.apache.kafka.connect.errors.ConnectException: java.lang.OutOfMemoryError: Java heap space
    at io.debezium.connector.mysql.MySqlConnectorTask.start(MySqlConnectorTask.java:275)
    at io.debezium.connector.common.BaseSourceTask.start(BaseSourceTask.java:49)
    at io.debezium.embedded.EmbeddedEngine.run(EmbeddedEngine.java:791)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
    at java.util.concurrent.FutureTask.run(FutureTask.java:266)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.OutOfMemoryError: Java heap space
    at java.util.Arrays.copyOf(Arrays.java:3332)
    at java.lang.AbstractStringBuilder.ensureCapacityInternal(AbstractStringBuilder.java:124)
    at java.lang.AbstractStringBuilder.append(AbstractStringBuilder.java:596)
    at java.lang.StringBuilder.append(StringBuilder.java:190)
    at java.io.BufferedReader.readLine(BufferedReader.java:358)
    at java.io.BufferedReader.readLine(BufferedReader.java:389)
    at java.nio.file.Files.readAllLines(Files.java:3205)
    at io.debezium.relational.history.FileDatabaseHistory.lambda$recoverRecords$2(FileDatabaseHistory.java:136)
    at io.debezium.relational.history.FileDatabaseHistory$$Lambda$181/230925968.run(Unknown Source)
    at io.debezium.util.FunctionalReadWriteLock.write(FunctionalReadWriteLock.java:96)
    at io.debezium.relational.history.FileDatabaseHistory.recoverRecords(FileDatabaseHistory.java:133)
    at io.debezium.relational.history.AbstractDatabaseHistory.recover(AbstractDatabaseHistory.java:73)
    at io.debezium.connector.mysql.MySqlSchema.loadHistory(MySqlSchema.java:252)
    at io.debezium.connector.mysql.MySqlTaskContext.loadHistory(MySqlTaskContext.java:165)
    at io.debezium.connector.mysql.MySqlConnectorTask.start(MySqlConnectorTask.java:105)
    ... 7 more

0 个答案:

没有答案