The ordering tension
A single ordered queue is easy to reason about but cannot scale, since one consumer processes it serially. To scale, brokers split a topic into many partitions that are consumed in parallel. The catch is that global order across partitions is lost.
Ordering keys to the rescue
Most systems offer per key order instead of global order. The producer attaches an ordering key, often an entity id like a user or account. The broker hashes the key to choose a partition, so:
- All messages with the same key land on the same partition and stay strictly ordered.
- Different keys spread across partitions and run in parallel.
This means events for one account are ordered relative to each other, which is usually all the business needs.
Things to watch
- Choose a key with high cardinality so load spreads evenly. A low cardinality key creates a hot partition.
- Repartitioning later can reorder in flight messages, so pick the key carefully up front.
Key idea
An ordering key routes all messages for one entity to one partition giving per key order while many partitions still scale throughput.