The cost of opening a connection
Each new TCP connection needs a handshake, and a secure one adds a TLS exchange on top. That can cost several round trips before any real data flows. Doing this per request wastes time and burns CPU.
Reuse
Keeping a connection open lets many requests share one setup.
- Keep alive holds a connection idle so the next request skips the handshake.
- Pipelining and multiplexing send several requests over one connection.
- Warm connections avoid slow start because the path is already probed.
Pooling
A connection pool keeps a set of ready connections that callers borrow and return.
- Bounded size caps how many connections hit a backend.
- Reuse spreads handshake cost across many calls.
- Health checks drop stale connections before they fail a request.
Sizing
Too few connections and callers queue, too many and the backend drowns in open sockets. Size the pool to the backend capacity, not the client count.
Key idea
Reuse connections through a bounded pool so handshakes happen rarely and latency drops on every request.