Stages connected by queues
A processing pipeline chains stages, each consuming from an inbound queue and producing to the next. When stages run at different speeds, the queue between a fast and a slow stage either grows unbounded or must push back.
The unbounded queue trap
An unbounded queue feels convenient but is dangerous. If a producer is faster than a consumer, the queue grows forever, latency climbs as items wait longer, and eventually memory is exhausted. The system fails not from logic errors but from accumulation.
Backpressure as a signal
Backpressure is the mechanism by which a slow consumer signals upstream to slow down. With a bounded queue:
- When the queue is full, the producer blocks or is told to wait.
- The producer naturally throttles to the consumer rate.
- Memory stays bounded and latency stays predictable.
Some systems instead apply load shedding, dropping or sampling items when overwhelmed, which trades completeness for stability.
Key idea
Backpressure replaces dangerous unbounded queues with bounded ones so a slow consumer forces a fast producer to throttle, keeping memory bounded and latency predictable instead of letting work pile up.