我将通过AWS MSK(Kafka)使用AWS Database Migration Service(DMS)。
我想将同一笔交易中的所有更改发送到Kafka主题的同一分区中-为了保证正确的消息顺序(引用完整性)
为此,我将启用以下属性:
IncludeTransactionDetails – Provides detailed transaction information from the source database. This information includes a commit timestamp, a log position, and values for transaction_id, previous_transaction_id, and transaction_record_id (the record offset within a transaction). The default is false. https://docs.aws.amazon.com/dms/latest/userguide/CHAP_Target.Kafka.html
另外,正如我从同一文档中看到的那样:
AWS DMS supports the following two forms for partition keys:
1. SchemaName.TableName: A combination of the schema and table name.
2. ${AttributeName}: The value of one of the fields in the JSON, or the primary key of the table in the source database.
我有一个问题-如果“ IncludeTransactionDetails = true”,我能否将事件JSON中的“ transaction_id”用作MSK(Kafka)迁移主题的分区键?
答案 0 :(得分:0)
文档说,您可以定义分区键来对数据进行分组 “您还为每个表定义了一个分区键,Apache Kafka使用该键将数据分组到其分区中”