Choosing the next backend
A load balancer must pick which backend handles each request. The algorithm it uses shapes how evenly load actually lands.
The common choices
- Round robin cycles through backends in order. Simple and fair when requests cost about the same, but blind to how busy each backend is.
- Least connections sends the next request to the backend with the fewest active connections. It adapts to uneven request costs and slow backends.
- Weighted variants give bigger machines a larger share, useful for mixed hardware.
- Random with two choices picks two backends at random and sends to the less loaded one, getting most of the benefit of least connections far more cheaply.
Matching algorithm to workload
Round robin shines for uniform, short requests. Least connections wins when request durations vary widely, since it avoids piling work on a backend stuck on a slow request. The power of two random choices is popular at scale because it needs little coordination yet avoids the worst hotspots.
Key idea
Round robin is simple and fair for uniform requests, while least connections and two random choices adapt better when request costs vary.