Two ways to shape traffic
Both algorithms control how fast requests pass, but they treat bursts differently.
The token bucket holds tokens up to a capacity and refills at a steady rate. Each request spends a token. If tokens have piled up during quiet time, a sudden burst can spend them all at once. So the token bucket allows bursts up to the bucket size while bounding the long run average.
The leaky bucket models a queue that drains at a fixed rate, like water leaking from a hole. Requests enter the bucket and leave at a constant pace regardless of how bursty the arrivals were. It smooths output into a steady stream and drops anything that overflows the queue.
Choosing between them
- Pick token bucket when occasional bursts are fine and you only care about the average rate, such as a public API.
- Pick leaky bucket when a downstream system needs a perfectly even feed, such as a device that cannot tolerate spikes.
Key idea
Token bucket permits bursts up to a cap while leaky bucket forces a smooth constant output rate.