The hidden ceiling
A database can only hold so many connections at once. Each one costs memory and a backend process or thread. Past a point, adding connections makes the database slower, not faster, because they fight over CPU and locks.
Why naive scaling breaks
If every app server opens its own connections, scaling the app tier multiplies connections fast.
- One hundred servers times fifty connections is five thousand, far past the limit.
- The database starts thrashing and rejecting new connections.
The fix is pooling
A connection pool keeps a small set of reusable connections and lends them out per query.
- Requests borrow a connection, use it briefly, and return it.
- A shared proxy pooler can cap total connections across all app servers.
- Tune the pool size to the database, not to the number of app servers.
The pool decouples app concurrency from database connection count.
Key idea
Cap and share database connections with a pool so the app tier can scale without overwhelming the database.