Where the time goes
A workload is CPU bound when its speed is limited by how fast the processor can compute, and IO bound when it spends most of its time waiting on input and output such as disk, network, or a database.
Matching the model to the bottleneck
The bottleneck decides the right concurrency tool:
- CPU bound work needs real parallelism across cores. Adding more threads than cores does not help, since the cores are already busy; it only adds switch overhead.
- IO bound work spends time waiting, so a single core can juggle many tasks while each waits. Async or many lightweight tasks shine here, because waiting tasks cost almost nothing.
A common mistake
Throwing thousands of threads at a CPU bound job wastes memory and adds context switches without speeding anything up. Conversely, using only as many threads as cores for an IO bound job leaves the CPU idle while threads sit blocked on the network. Knowing which kind you have guides whether to parallelize or to overlap waiting.
Key idea
CPU bound work needs parallelism up to the core count while IO bound work benefits from overlapping many waiting tasks, so identifying the bottleneck guides the concurrency choice.