Two classic algorithms
Both limit a rate, but they shape traffic differently.
Leaky bucket
Requests enter a queue that drains at a fixed rate, like water leaking from a bucket. Output is perfectly smooth, and if the bucket overflows, excess requests are dropped.
- Strength: a steady, predictable output rate with no bursts.
- Weakness: it cannot reward an idle client with a burst, and adds queuing delay.
Token bucket
Tokens are added to a bucket at a fixed rate up to a cap. Each request spends a token; if the bucket is empty the request is rejected or waits.
- Strength: a client that has been idle accumulates tokens and may burst up to the bucket size.
- Weakness: short bursts can exceed the steady average rate.
Choosing between them
- Leaky bucket suits a downstream that needs a smooth, even feed.
- Token bucket suits APIs that want to tolerate bursty but bounded traffic.
Key idea
A leaky bucket enforces a smooth constant output while a token bucket allows saved up bursts, so pick by whether your downstream prefers steady flow or can absorb controlled spikes.