Thread Pools
Creating a new thread for every task is wasteful because thread creation costs memory and time. A thread pool keeps a fixed set of worker threads alive and feeds them tasks from a shared work queue.
How it works:
- Tasks are submitted to a queue rather than spawning threads directly.
- Idle workers pull the next task, run it, then loop back for more.
- When all workers are busy, new tasks wait in the queue.
Benefits include bounded concurrency, which prevents the system from being overwhelmed by thousands of threads, and reuse, which removes repeated creation cost. Tuning the pool size matters: too few threads underuse the CPU, while too many cause excessive context switching.
A common rule is to size a CPU bound pool near the core count, and a larger pool for IO bound work where threads spend time waiting.
Watch for queue growth: an unbounded queue can hide a system that cannot keep up.
Key idea
A thread pool reuses a fixed set of workers pulling from a queue, giving bounded concurrency and avoiding repeated thread creation cost.