The fast producer problem
In a realtime stream a producer may emit faster than a consumer can process. Without control, messages pile into a queue that grows without bound until memory is exhausted and the process crashes. Backpressure is the mechanism that pushes back on the producer.
How backpressure flows
- The consumer signals how much it can accept, often through a bounded buffer.
- When the buffer fills, the producer is told to pause or slow down.
- As the consumer drains, it signals that it can resume.
Strategies under overload
- Block the producer until space frees, which propagates pressure upstream.
- Drop the oldest or newest messages when staleness is acceptable.
- Sample by keeping only the latest value, ideal for things like cursor positions.
Why it matters
Backpressure turns an unbounded failure into a bounded and controlled degradation. The system either slows gracefully or sheds load deliberately rather than crashing unpredictably.
Key idea
Backpressure signals a producer to slow when a consumer falls behind, replacing unbounded queue growth and crashes with bounded buffers and deliberate strategies to block, drop, or sample.