One equation, broad reach
Little's law states that the average number of items in a system equals the arrival rate times the average time each spends inside it. Written simply, L equals lambda times W.
Reading the terms
- L is the average number of requests in the system at once, the concurrency.
- lambda is the arrival rate, requests per second.
- W is the average time a request spends in the system.
It holds for any stable system regardless of the arrival pattern, which makes it remarkably general.
How to use it
- Need concurrency: multiply throughput by latency. At 1000 QPS and 50 milliseconds each, about 50 requests are in flight on average.
- This sizes thread pools and connection pools, which must hold at least that many slots.
A pool smaller than the law predicts becomes the bottleneck itself, queueing requests that the backend could otherwise have served.
Key idea
Little's law ties concurrency to arrival rate times latency, sizing thread and connection pools for any stable system.