Topics split into partitions
A Kafka topic is a named stream of records. Each topic is divided into partitions, and each partition is an ordered append only log on disk.
Why partitions exist
- They let a topic spread across many brokers, so throughput scales beyond one machine.
- They allow parallel consumption, since each partition can be read by a separate consumer.
Ordering is per partition
Kafka guarantees order within a partition, not across the whole topic. To keep related records ordered, the producer picks a key. All records with the same key hash to the same partition, so they stay in order.
Choosing a key
If you key by user id, every event for one user lands in one partition in order. A null key spreads records round robin for even load but gives up per key ordering.
Key idea
Partitions are the unit of scale and ordering in Kafka, and a record key decides which partition keeps related records in order.