Question

根据Kinesis文档，序列号应该是唯一的，但是我们看到同一值可在多个记录中重用。我们的事件生产者是内部使用KPL的Spring Boot应用程序，消费者是AWS lambda。在测试过程中，我们执行了几次重新分片。在下面，您可以看到多次重复使用的示例序列号。怎么可能？

"Records": [{
    "kinesis": {
        "kinesisSchemaVersion": "1.0",
        "partitionKey": "00000000000000002",
        "sequenceNumber": "49596124085897508159438713510240079964989152308217511954",
        "data": "************************",
        "approximateArrivalTimestamp": 1558991793.009
    },
    "eventSource": "aws:kinesis",
    "eventVersion": "1.0",
    "eventID": "shardId-000000000001:49596124085897508159438713510240079964989152308217511954",
    "eventName": "aws:kinesis:record",
    "invokeIdentityArn": "-----------------",
    "awsRegion": "us-east-1",
    "eventSourceARN": "-----------------"
}, {
    "kinesis": {
        "kinesisSchemaVersion": "1.0",
        "partitionKey": "00000000000000003",
        "sequenceNumber": "49596124085897508159438713510240079964989152308217511954",
        "data": ""************************",",
        "approximateArrivalTimestamp": 1558991793.009
    },
    "eventSource": "aws:kinesis",
    "eventVersion": "1.0",
    "eventID": "shardId-000000000001:49596124085897508159438713510240079964989152308217511954",
    "eventName": "aws:kinesis:record",
    "invokeIdentityArn": "-----------------",
    "awsRegion": "us-east-1",
    "eventSourceARN": "-----------------"
}, {
    "kinesis": {
        "kinesisSchemaVersion": "1.0",
        "partitionKey": "00000000000000004",
        "sequenceNumber": "49596124085897508159438713510240079964989152308217511954",
        "data": ""************************",",
        "approximateArrivalTimestamp": 1558991793.009
    },
    "eventSource": "aws:kinesis",
    "eventVersion": "1.0",
    "eventID": "shardId-000000000001:49596124085897508159438713510240079964989152308217511954",
    "eventName": "aws:kinesis:record",
    "invokeIdentityArn": "-----------------",
    "awsRegion": "us-east-1",
    "eventSourceARN": "-----------------"
}]

Answer 1

当Kinesis流作者将KPL与用户记录聚合一起使用时（请参见Consumer De-aggregation），用户记录将一起批处理并作为单个Kinesis记录交付给常规Kinesis消费者。 Kinesis记录序列号在这种情况下是唯一的，但是我们需要实现解聚合。

但是，如果为Lambda启用了增强的扇出功能，则用户记录将作为单独的Kinesis记录交付（不需要解聚集），并且它们共享相同的序号。

因此Kinesis记录序列号并不总是唯一的。

Kinesis创建具有相同序列号的多条记录

1 个答案: