The dual write problem
A service often must update its database and publish an event. Doing these as two separate operations is unsafe: a crash between them can commit the database change but lose the event, or send the event without committing. There is no atomicity across two systems.
The outbox pattern
- In the same database transaction as the business change, insert a row into an outbox table.
- A separate relay reads new outbox rows and publishes them to the message broker, then marks them sent.
- Because the business change and the outbox insert commit together, an event is never lost or sent without its data.
How the relay reads
- Polling: the relay queries unsent outbox rows on an interval.
- Change data capture: the relay tails the database log and streams new outbox rows with low latency.
The inbox pattern
The relay may publish a message more than once, so consumers need dedup. The inbox is a table where the consumer records each processed message id inside the same transaction as its effect. A repeat with a known id is skipped, making the consumer idempotent.
Key idea
The outbox writes the event and the data in one transaction so publishing is atomic, and the inbox records processed ids so consumers safely dedupe the at least once delivery that results.