The original challenge
The c10k problem named the goal of handling ten thousand concurrent connections on a single server. The classic thread per connection design hits a wall well before that. Each thread needs a stack of perhaps a megabyte, and ten thousand threads mean gigabytes of memory plus a scheduler drowning in context switches.
Where the cost goes
- Memory grows linearly with connection count because of per thread stacks
- The scheduler spends more time switching than working as thread counts rise
- Most connections are idle at any instant, so most threads just wait
The insight is that connections vastly outnumber the threads truly doing work at any moment. You do not need a thread per connection, only enough threads to handle the active ones.
The event driven answer
The solution pairs non blocking sockets with an efficient readiness interface like epoll. A small number of threads, often one per core, run event loops that multiplex thousands of connections each. Memory now scales with connections as cheap descriptor state rather than expensive stacks, and idle connections cost almost nothing.
Key idea
The c10k problem showed that thread per connection scales poorly, so high concurrency servers use a few event loops over non blocking sockets to multiplex many connections cheaply.