我的带有随机分区键的Kinesis Producer将记录放置在shard0中。我希望从我的使用者的分片0中读取它。
我正在观察的是,对于某些记录,情况并非如此:
理想情况:放入000000000002并从000000000002中读取
2019-02-12 11:18:52.278 INFO 10152 --- [TaskExecutor-60] c.i.i.e.kinesis.KinesisProducer : Put Result={ShardId: shardId-000000000002,SequenceNumber: 49592292474998443170974666693116643662109230963259080738,}
Shard: shardId-000000000002, PartitionKey: partitionKey-MTALK__22a5e920-014f-45b0-8d27-aa3f511d3738, SequenceNumber: 49592292474998443170974666693116643662109230963259080738
观测值:放入000000000000并从000000000002中读取
Shard: shardId-000000000002, PartitionKey: partitionKey-MTALK__09241131-5f81-44fc-852f-7f6483b5e18b, SequenceNumber: 49592292458897305137635556784925437217617884560152854530
此外,尽管检查点和Trim Horizon正确,但我看到消费者重新启动时再次使用了相同的消息-来自不同堆的相同分区密钥和相同序列号。
Put Result={ShardId: shardId-000000000003,SequenceNumber: 49592278123331569911393613339504152562720511055896772658,}
Consumed as:
Shard: shardId-000000000002, PartitionKey: partitionKey-MTALK__3f1caefc-e072-4854-8cb8-f42af2cb67e2, SequenceNumber: 49592278123331569911393613339504152562720511055896772658
Shard: shardId-000000000003, PartitionKey: partitionKey-MTALK__3f1caefc-e072-4854-8cb8-f42af2cb67e2, SequenceNumber: 49592278123331569911393613339504152562720511055896772658
Shard: shardId-000000000003, PartitionKey: partitionKey-MTALK__3f1caefc-e072-4854-8cb8-f42af2cb67e2, SequenceNumber: 49592278123331569911393613339504152562720511055896772658