Consumer groups share work
A consumer group is a set of consumers that together read a topic. Kafka assigns each partition to exactly one consumer in the group, so the group splits the work without double reading.
What triggers a rebalance
A rebalance reassigns partitions when membership changes:
- A consumer joins the group.
- A consumer leaves or crashes and misses heartbeats.
- The topic gains new partitions.
The stop the world cost
In the classic protocol, a rebalance pauses all consumers in the group while the group coordinator computes a new assignment. During this pause no records are processed, which can hurt latency for large groups.
Reducing the pain
- Cooperative rebalancing moves only the partitions that must change, instead of revoking everything.
- Static membership gives each consumer a stable id so a quick restart does not trigger a full reshuffle.
Key idea
A rebalance redistributes partitions when a consumer group changes, and cooperative or static membership limits the costly pause.