Kafka Topic ordering when scaling up the partitions

Question

Consider your producers create messages for the users of a system and the order of them is important in the user level.

My producers, add messages to the topic which have two partitions and I am using hashing against the user_id to put all the messages of each user in the same partition to guarantee the order.

How can I scale up the system and add more partitions to the topic while keeping the order of the messages?

How Kafka treat the messages that are already produced before partitioning? What will happen to the messages that consume but not committed back to the Kafka to update the offset?

Answer 1

1。在消费者客户端使用树集（有序集）缓存消息，保持1分钟（或更少）； kafka只保证一个分区的顺序，我想生产者也不能保证顺序。

2。如果不手动提交偏移量，则在下一个获取请求中，将得到相同的消息。无论如何，在消费者客户处，即使您已同意使用偏移量，也应确保消息的幂等性。