The Dual Write Problem
A service often must do two things on a change: update its database and publish an event to a message broker. If it writes the database and then publishes separately, a crash between the two leaves them inconsistent. The data changed but no event went out, or an event fired for a write that rolled back.
The Outbox Pattern
- Add an outbox table in the same database.
- In the same transaction as the business change, insert a row describing the event into the outbox.
- Because both writes share one transaction, they commit or abort together atomically.
Getting Events Out
A separate process reads new outbox rows and publishes them to the broker, then marks them as sent. This relay can use polling or change data capture that tails the database log.
Delivery Semantics
- Delivery is at least once, since a crash after publish but before marking sent causes a resend.
- Consumers must be idempotent, typically deduplicating on an event id.
Key idea
The outbox pattern writes events to a table in the same transaction as the data, then relays them, giving atomic at least once delivery without a distributed transaction.