Contention and Scalability
Contention happens when multiple threads compete for the same shared resource, most often a lock. Under contention only one thread makes progress while the others wait, so adding more threads or cores stops helping and can even slow the system down. Contention is the practical reason real programs fall short of the speedup Amdahl law predicts.
The damage shows up in several ways:
- A hot lock serializes work, turning parallel code back into sequential code
- Waiting threads incur context switch overhead as they block and wake
- On many cores, frequently written shared variables cause cache line traffic as the line bounces between processors, a problem worsened by false sharing
Techniques to reduce contention:
- Reduce lock scope Hold locks for the shortest possible time and do not perform slow work like input or output while holding one
- Lock splitting and striping Replace one lock guarding everything with several locks guarding independent parts, so unrelated operations no longer collide
- Lock free and atomic operations Use compare and swap based structures that coordinate without blocking, though they can suffer their own contention under heavy write load
- Thread local or sharded state Give each thread its own copy and combine results later, eliminating sharing on the hot path
- Read mostly structures Use immutable snapshots or readers writer locks when reads dominate
The goal of a scalable design is to shrink the serial, contended portion so that throughput keeps rising as you add hardware.
Key idea
Contention on a hot lock serializes work and stalls scaling, so good designs shrink lock scope, split or shard state, and use atomic or read mostly structures to let throughput grow with cores.