A law that always holds
Little law says that in a stable system the average number of items in flight equals the arrival rate times the average time each item spends in the system. Written for a server:
- concurrency equals throughput times latency
It holds for any stable system regardless of internal detail, which makes it a powerful sanity check.
Sizing threads with it
If you need a throughput of two thousand requests per second and each request takes fifty milliseconds, then the average in flight count is two thousand times zero point zero five, which is one hundred. You need at least one hundred slots of concurrency in flight, whether that is threads or async tasks.
Spotting impossible plans
- If a pool has fifty threads but each request takes one hundred milliseconds, the most throughput it can sustain is fifty divided by zero point one, which is five hundred per second.
- Asking it for two thousand per second is impossible without either more concurrency or lower latency.
The law turns hand waving into arithmetic.
Key idea
Little law ties concurrency to throughput times latency, so you can compute the in flight count a target load demands instead of guessing.