Is manipulating the "read-offset" as kafka consumer bad-practice?

Question

We have an ongoing discussion about the correct (or intended) usage of Kafka for events.

The arguing point is the ability of a consumer to not only subscribe (or resubscribe) to a topic but also to modify its own read offset.

Am I right in saying that "A consumer should be design in a way that it never modifies its own read offset!"

Reasoning behind this:

• The consumer cannot know what events actually are stored inside a topic (log retention)
• ... So restoring a complete state from "delta"-events is not possible.
• The consumer has consumed an event once and confirmed this to the broker. why consuming again?

Answer 1

如果您的使用者实例属于相同的使用者组，则使用者无需保持主题的读取状态。读取状态仅是主题的偏移，到目前为止，您的消费者尚未读取的记录。如果您的主题具有多个分区，则属于同一消费者组的消费者可以在消费者之间分配工作负载。如果某个使用者崩溃或发生故障，则来自同一使用者组的其他使用者将知道他们从哪个分区偏移量继续使用记录。