A poison message stalls the stream
Some records cannot be processed: malformed data, a missing reference, or a bug. If a consumer keeps retrying the same record, it blocks the partition and falls behind for everyone.
Routing to a dead letter topic
A dead letter topic is a separate topic where a consumer sends records it has failed to process after a retry limit. The consumer then commits past the bad record and keeps moving.
What to capture
Include enough context to diagnose later:
- The original payload and its key.
- The error and a stack trace or reason.
- The source topic, partition, and offset.
- A timestamp and retry count.
Operating the topic
Teams monitor the dead letter topic and alert when it grows. Operators inspect records, fix the cause, and may replay corrected records back into the main flow.
Key idea
A dead letter topic parks records that keep failing, letting the consumer commit past them and stay flowing while operators inspect and replay later.