Object Pool
Object Pool isn't one of the 23 GoF patterns, but it's one of the most consequential creational patterns in production systems. Every Java service talking to a database goes through a connection pool, every web server runs requests on a thread pool, and most HTTP clients keep a pool of open sockets. The pattern is simple to state: keep a bounded set of expensive objects alive, lend them to clients on request, and take them back when the clients are done, instead of creating and destroying one per use.
It also has a reputation problem. In the early days of Java, developers pooled everything, including small ordinary objects, to "help" the garbage collector. On modern runtimes that usually makes programs slower and buggier. This page covers both sides: how to build a correct pool (bounded capacity, borrow and return, validation, reset, timeouts, leak handling and sizing) and how to tell a resource worth pooling from an object that should simply be allocated and forgotten. "Design a connection pool" is also a classic LLD interview question, and the implementation here is a good answer to it.
| Category | Creational (not in the GoF catalogue) |
| Also known as | Resource pool |
| Solves | Expensive creation, and a hard limit on how many instances may exist at once |
| Main cost | Lifecycle bugs: leaks, state bleeding between users, stale objects, deadlocks on exhaustion |
| Rule of thumb | Pool resources (connections, threads, large native buffers), not objects |
Intent
Manage a set of initialised, reusable objects. Clients borrow an object from the pool, use it exclusively, and return it. The pool creates objects when needed up to a limit, makes clients wait (with a timeout) when all are in use, and checks or resets objects between borrowers.
Two separate motivations hide in that intent, and a good pool serves both:
- Amortising creation cost. Opening a database connection involves a TCP handshake, often a TLS handshake, authentication and session setup. Creating a thread involves the operating system. Doing that once and reusing the result saves a lot of time per request.
- Bounding concurrency. A database accepts only so many connections. A machine can run only so many threads usefully. The pool's maximum size is a throttle that protects the downstream resource, and that matters as much as the reuse.
The problem
A report service opens a fresh database connection for every query. It works in development. In production, two things go wrong.
import java.util.concurrent.atomic.AtomicInteger;
final class FakeDatabase {
final AtomicInteger handshakes = new AtomicInteger();
final AtomicInteger open = new AtomicInteger();
final int maxConnections = 100;
Connection connect() {
handshakes.incrementAndGet(); // TCP + TLS + auth: milliseconds each, in reality
if (open.incrementAndGet() > maxConnections) {
open.decrementAndGet();
throw new IllegalStateException("FATAL: too many connections");
}
return new Connection(this);
}
}
final class Connection implements AutoCloseable {
private final FakeDatabase db;
Connection(FakeDatabase db) { this.db = db; }
int query(String sql) { return 42; }
public void close() { db.open.decrementAndGet(); }
}
final class ReportService {
private final FakeDatabase db;
ReportService(FakeDatabase db) { this.db = db; }
int revenue() {
try (Connection c = db.connect()) { // a new connection per call
return c.query("SELECT sum(amount) FROM orders");
}
}
}
public class Before {
public static void main(String[] args) {
FakeDatabase db = new FakeDatabase();
ReportService reports = new ReportService(db);
for (int i = 0; i < 1000; i++) reports.revenue();
if (db.handshakes.get() != 1000) throw new AssertionError(db.handshakes.get());
System.out.println("1000 queries cost " + db.handshakes.get() + " connection handshakes");
}
}
class FakeDatabase:
def __init__(self, max_connections: int = 100) -> None:
self.handshakes = 0
self.open = 0
self.max_connections = max_connections
def connect(self) -> "Connection":
self.handshakes += 1 # TCP + TLS + auth, in reality
if self.open >= self.max_connections:
raise RuntimeError("FATAL: too many connections")
self.open += 1
return Connection(self)
class Connection:
def __init__(self, db: FakeDatabase) -> None:
self._db = db
def query(self, sql: str) -> int:
return 42
def close(self) -> None:
self._db.open -= 1
def revenue(db: FakeDatabase) -> int:
conn = db.connect() # a new connection per call
try:
return conn.query("SELECT sum(amount) FROM orders")
finally:
conn.close()
db = FakeDatabase()
for _ in range(1000):
revenue(db)
assert db.handshakes == 1000
print(f"1000 queries cost {db.handshakes} connection handshakes")
- Every query pays the full connection cost. One handshake per query. With a remote, TLS-protected database, setup can easily take longer than a simple query, so most of the request's time is spent connecting.
- Nothing limits how many connections exist. Under a traffic spike, each concurrent request opens its own connection. Run twenty instances of the service and the total can blow through the database's limit (PostgreSQL's
max_connectionsdefaults to 100). At that point, new connections fail with "too many connections", and the database itself slows down under the per-connection memory and process overhead. Unbounded creation turns a traffic spike into an outage.
A pool fixes both: connections are created once and reused, and at most N exist at a time, however many requests arrive.
Structure
classDiagram
class ObjectPool {
-maxSize int
-idle Deque
-leased Set
-permits Semaphore
+borrow(timeout) Lease
-release(obj, broken)
+close()
}
class Lease {
+get() T
+invalidate()
+close()
}
class PooledObjectFactory {
<<interface>>
+create() T
+validate(obj) boolean
+reset(obj)
+destroy(obj)
}
class Client
class Connection
Client --> ObjectPool : borrows from
Client --> Lease : uses and closes
ObjectPool --> PooledObjectFactory : creates, checks, resets
ObjectPool o-- Connection : idle and leased
Lease --> Connection : wraps
PooledObjectFactory ..> Connection : creates
The pool owns the objects and their bookkeeping. A factory knows the resource-specific parts: how to create one, how to tell if it's still usable, how to wipe the previous borrower's state, and how to destroy it. Clients receive a lease, a small handle whose close() returns the object. Returning through a handle rather than calling pool.release(obj) lets the language's resource-cleanup constructs (try-with-resources, with, finally) do the returning, which prevents most leaks.
Participants
- Pool. Holds idle objects, tracks leased ones, enforces the maximum size, makes borrowers wait up to a timeout, and decides on return whether an object is reused or destroyed.
- Reusable object (
Connection). The expensive resource itself. It shouldn't need to know it's pooled. - Factory / lifecycle policy.
create,validate,resetanddestroyfor one kind of resource. Separating this from the pool makes the pool generic. - Lease (optional but recommended). A per-borrow handle. It returns the object exactly once, and can mark it broken so it's destroyed instead of reused.
- Client. Borrows, uses and returns, ideally inside a construct that guarantees return.
Implementation
The three pools below implement the same design:
- capacity is a counting semaphore with
maxSizepermits, so at mostmaxSizeobjects are ever lent out or idle; - idle objects are reused last in, first out, so the most recently used (warmest, least likely to have timed out) object goes out first;
- objects are validated on borrow, and broken ones are destroyed and replaced;
- objects are reset on return, so the next borrower never sees the previous one's state;
- borrowing times out instead of waiting forever;
- a returned object must be one this pool lent out, and each lease returns its object only once.
import java.util.*;
import java.util.concurrent.*;
import java.util.concurrent.atomic.*;
interface PooledObjectFactory<T> {
T create();
boolean validate(T obj); // still usable? (e.g. Connection.isValid(timeout))
void reset(T obj); // wipe per-borrower state before reuse
void destroy(T obj);
}
final class PoolExhaustedException extends RuntimeException {
PoolExhaustedException(String message) { super(message); }
}
final class BoundedPool<T> implements AutoCloseable {
private final PooledObjectFactory<T> factory;
private final Semaphore permits;
private final Deque<T> idle = new ConcurrentLinkedDeque<>();
// Identity, not equals(): two equal-looking objects are still different resources.
private final Set<T> leased = Collections.synchronizedSet(Collections.newSetFromMap(new IdentityHashMap<>()));
private volatile boolean closed;
BoundedPool(PooledObjectFactory<T> factory, int maxSize) {
if (maxSize < 1) throw new IllegalArgumentException("maxSize must be >= 1");
this.factory = factory;
this.permits = new Semaphore(maxSize, true); // fair: waiters are served in order
}
final class Lease implements AutoCloseable {
private final T obj;
private final AtomicBoolean returned = new AtomicBoolean();
private volatile boolean broken;
private Lease(T obj) { this.obj = obj; }
T get() {
if (returned.get()) throw new IllegalStateException("lease already returned");
return obj;
}
void invalidate() { broken = true; } // destroy instead of reuse
@Override public void close() {
if (returned.compareAndSet(false, true)) release(obj, broken);
}
}
Lease borrow(long timeout, TimeUnit unit) throws InterruptedException {
if (closed) throw new IllegalStateException("pool is closed");
if (!permits.tryAcquire(timeout, unit))
throw new PoolExhaustedException("no object available within " + timeout + " " + unit);
try {
T obj;
while ((obj = idle.pollFirst()) != null && !factory.validate(obj)) {
factory.destroy(obj); // stale: drop it and try the next one
}
if (obj == null) obj = factory.create();
leased.add(obj);
return new Lease(obj);
} catch (RuntimeException e) {
permits.release(); // creation failed: give the slot back
throw e;
}
}
private void release(T obj, boolean broken) {
if (!leased.remove(obj)) throw new IllegalStateException("object was not borrowed from this pool");
try {
if (broken || closed) {
factory.destroy(obj);
} else {
factory.reset(obj);
idle.offerFirst(obj); // LIFO: reuse the warmest object first
}
} catch (RuntimeException e) {
factory.destroy(obj); // reset failed: never hand it out again
} finally {
permits.release();
}
}
int idleCount() { return idle.size(); }
int leasedCount() { return leased.size(); }
@Override public void close() {
closed = true;
T obj;
while ((obj = idle.pollFirst()) != null) factory.destroy(obj);
// Leased objects are destroyed as they come back.
}
}
// ---- a fake connection and its factory, for the demo ----------------------
final class FakeConnection {
final int id;
boolean broken;
String tenant; // per-session state that must not leak
FakeConnection(int id) { this.id = id; }
}
final class FakeConnectionFactory implements PooledObjectFactory<FakeConnection> {
final AtomicInteger created = new AtomicInteger();
final AtomicInteger destroyed = new AtomicInteger();
public FakeConnection create() { return new FakeConnection(created.incrementAndGet()); }
public boolean validate(FakeConnection c) { return !c.broken; }
public void reset(FakeConnection c) { c.tenant = null; }
public void destroy(FakeConnection c) { destroyed.incrementAndGet(); }
}
public class Main {
static void check(boolean condition, String message) { if (!condition) throw new AssertionError(message); }
public static void main(String[] args) throws Exception {
FakeConnectionFactory factory = new FakeConnectionFactory();
BoundedPool<FakeConnection> pool = new BoundedPool<>(factory, 4);
// 16 threads x 50 borrows: never more than 4 in use, never more than 4 created.
AtomicInteger inUse = new AtomicInteger(), peak = new AtomicInteger();
ExecutorService workers = Executors.newFixedThreadPool(16);
List<Future<?>> jobs = new ArrayList<>();
for (int t = 0; t < 16; t++) {
jobs.add(workers.submit(() -> {
for (int i = 0; i < 50; i++) {
try (var lease = pool.borrow(2, TimeUnit.SECONDS)) {
peak.accumulateAndGet(inUse.incrementAndGet(), Math::max);
lease.get().tenant = "tenant-" + Thread.currentThread().threadId();
inUse.decrementAndGet();
}
}
return null;
}));
}
for (Future<?> job : jobs) job.get();
workers.shutdown();
check(peak.get() <= 4, "peak in use " + peak.get());
check(factory.created.get() <= 4, "created " + factory.created.get());
// Reset: the next borrower never sees the previous tenant.
try (var lease = pool.borrow(1, TimeUnit.SECONDS)) { check(lease.get().tenant == null, "state leaked"); }
// Validation: a broken connection is replaced on the next borrow.
for (int i = 0; i < 4; i++) { // invalidate four borrows in a row
try (var lease = pool.borrow(1, TimeUnit.SECONDS)) { lease.invalidate(); }
}
check(factory.destroyed.get() == 4 && pool.idleCount() == 0, "invalidated objects are destroyed");
int before = factory.created.get();
try (var lease = pool.borrow(1, TimeUnit.SECONDS)) {
check(!lease.get().broken && factory.created.get() == before + 1, "fresh object created");
}
// Exhaustion: with all 4 borrowed, a fifth borrower times out instead of hanging.
List<BoundedPool<FakeConnection>.Lease> held = new ArrayList<>();
for (int i = 0; i < 4; i++) held.add(pool.borrow(1, TimeUnit.SECONDS));
try {
pool.borrow(50, TimeUnit.MILLISECONDS);
throw new AssertionError("expected exhaustion");
} catch (PoolExhaustedException expected) { }
// Double return is harmless; using a returned lease is an error.
var first = held.get(0);
first.close();
first.close();
check(pool.leasedCount() == 3, "returned exactly once");
try { first.get(); throw new AssertionError("use after return"); } catch (IllegalStateException expected) { }
held.forEach(lease -> lease.close());
pool.close();
System.out.println("peak " + peak.get() + ", created " + factory.created.get() + ", destroyed " + factory.destroyed.get());
}
}
import threading
import time
from collections import deque
from collections.abc import Callable, Iterator
from concurrent.futures import ThreadPoolExecutor
from contextlib import contextmanager
from dataclasses import dataclass
from typing import Generic, TypeVar
T = TypeVar("T")
class PoolExhausted(Exception):
pass
@dataclass
class Lifecycle(Generic[T]):
create: Callable[[], T]
validate: Callable[[T], bool]
reset: Callable[[T], None]
destroy: Callable[[T], None]
class BoundedPool(Generic[T]):
def __init__(self, lifecycle: Lifecycle[T], max_size: int) -> None:
if max_size < 1:
raise ValueError("max_size must be >= 1")
self._life = lifecycle
self._permits = threading.BoundedSemaphore(max_size)
self._lock = threading.Lock()
self._idle: deque[T] = deque()
self._leased: dict[int, T] = {} # keyed by id(): identity, not equality
self._closed = False
@contextmanager
def lease(self, timeout: float = 1.0) -> Iterator[T]:
obj = self._borrow(timeout)
broken = False
try:
yield obj
except BaseException:
broken = True # an error mid-use: don't trust the object
raise
finally:
self._release(obj, broken)
def _borrow(self, timeout: float) -> T:
if self._closed:
raise RuntimeError("pool is closed")
if not self._permits.acquire(timeout=timeout):
raise PoolExhausted(f"no object available within {timeout}s")
try:
while True:
with self._lock:
obj = self._idle.pop() if self._idle else None # LIFO
if obj is None:
obj = self._life.create()
break
if self._life.validate(obj):
break
self._life.destroy(obj) # stale: drop it and look again
with self._lock:
self._leased[id(obj)] = obj
return obj
except BaseException:
self._permits.release()
raise
def _release(self, obj: T, broken: bool) -> None:
with self._lock:
if self._leased.pop(id(obj), None) is None:
raise RuntimeError("object was not borrowed from this pool")
try:
if broken or self._closed:
self._life.destroy(obj)
else:
self._life.reset(obj)
with self._lock:
self._idle.append(obj)
except Exception:
self._life.destroy(obj)
finally:
self._permits.release()
@property
def idle_count(self) -> int:
return len(self._idle)
def close(self) -> None:
self._closed = True
with self._lock:
while self._idle:
self._life.destroy(self._idle.pop())
# ---- demo ------------------------------------------------------------------
class FakeConnection:
def __init__(self, ident: int) -> None:
self.id = ident
self.broken = False
self.tenant: str | None = None
created, destroyed = [0], [0]
counter_lock = threading.Lock()
def create() -> FakeConnection:
with counter_lock:
created[0] += 1
return FakeConnection(created[0])
def destroy(c: FakeConnection) -> None:
with counter_lock:
destroyed[0] += 1
def reset(c: FakeConnection) -> None:
c.tenant = None
pool = BoundedPool(Lifecycle(create, lambda c: not c.broken, reset, destroy), max_size=4)
in_use, peak = [0], [0]
def work(n: int) -> None:
for _ in range(50):
with pool.lease(timeout=2) as conn:
with counter_lock:
in_use[0] += 1
peak[0] = max(peak[0], in_use[0])
conn.tenant = f"tenant-{n}"
time.sleep(0.001) # hold it briefly so borrowers overlap
with counter_lock:
in_use[0] -= 1
with ThreadPoolExecutor(max_workers=16) as ex:
list(ex.map(work, range(16)))
assert peak[0] <= 4 and created[0] <= 4, (peak, created)
with pool.lease() as conn:
assert conn.tenant is None # reset on return
try: # an exception inside the block discards the object
with pool.lease() as conn:
raise ValueError("query failed")
except ValueError:
pass
assert destroyed[0] == 1
for _ in range(4): # break every idle connection
with pool.lease() as conn:
conn.broken = True
with pool.lease() as conn:
assert not conn.broken # validation replaced the broken one
held = [pool._borrow(1) for _ in range(4)]
try:
pool._borrow(0.05)
raise AssertionError("expected exhaustion")
except PoolExhausted:
pass
for c in held:
pool._release(c, broken=False)
try:
pool._release(held[0], broken=False) # double return is detected
raise AssertionError("expected double-return error")
except RuntimeError:
pass
pool.close()
print(f"peak {peak[0]}, created {created[0]}, destroyed {destroyed[0]}")
import assert from "node:assert/strict";
interface Lifecycle<T> {
create(): Promise<T>;
validate(obj: T): boolean;
reset(obj: T): void;
destroy(obj: T): void;
}
class PoolExhaustedError extends Error {}
// A counting semaphore for async code: waiters queue up and may time out.
class Semaphore {
#permits: number;
readonly #waiters: Array<() => void> = [];
constructor(permits: number) { this.#permits = permits; }
acquire(timeoutMs: number): Promise<void> {
if (this.#permits > 0) { this.#permits--; return Promise.resolve(); }
return new Promise((resolve, reject) => {
const wake = (): void => { clearTimeout(timer); resolve(); };
const timer = setTimeout(() => {
const i = this.#waiters.indexOf(wake);
if (i >= 0) this.#waiters.splice(i, 1);
reject(new PoolExhaustedError(`no object available within ${timeoutMs}ms`));
}, timeoutMs);
this.#waiters.push(wake);
});
}
release(): void {
const next = this.#waiters.shift();
if (next) next(); // hand the permit straight to the next waiter
else this.#permits++;
}
}
class BoundedPool<T extends object> {
readonly #idle: T[] = [];
readonly #leased = new Set<T>(); // Set uses identity for objects
readonly #permits: Semaphore;
#closed = false;
constructor(private readonly life: Lifecycle<T>, maxSize: number) {
if (maxSize < 1) throw new Error("maxSize must be >= 1");
this.#permits = new Semaphore(maxSize);
}
async acquire(timeoutMs = 1000): Promise<T> {
if (this.#closed) throw new Error("pool is closed");
await this.#permits.acquire(timeoutMs);
try {
let obj = this.#idle.pop(); // LIFO
while (obj !== undefined && !this.life.validate(obj)) {
this.life.destroy(obj);
obj = this.#idle.pop();
}
obj ??= await this.life.create();
this.#leased.add(obj);
return obj;
} catch (e) {
this.#permits.release();
throw e;
}
}
release(obj: T, broken = false): void {
if (!this.#leased.delete(obj)) throw new Error("object was not borrowed from this pool");
try {
if (broken || this.#closed) this.life.destroy(obj);
else { this.life.reset(obj); this.#idle.push(obj); }
} catch {
this.life.destroy(obj);
} finally {
this.#permits.release();
}
}
// The leak-proof way to borrow: the object always goes back, even on error.
async use<R>(fn: (obj: T) => Promise<R>, timeoutMs?: number): Promise<R> {
const obj = await this.acquire(timeoutMs);
let broken = false;
try {
return await fn(obj);
} catch (e) {
broken = true;
throw e;
} finally {
this.release(obj, broken);
}
}
get idleCount(): number { return this.#idle.length; }
close(): void {
this.#closed = true;
for (const obj of this.#idle.splice(0)) this.life.destroy(obj);
}
}
// ---- demo ------------------------------------------------------------------
class FakeConnection {
broken = false;
tenant: string | undefined;
constructor(readonly id: number) {}
}
let created = 0, destroyed = 0;
const pool = new BoundedPool<FakeConnection>({
create: async () => new FakeConnection(++created),
validate: (c) => !c.broken,
reset: (c) => { c.tenant = undefined; },
destroy: () => { destroyed++; },
}, 3);
const sleep = (ms: number) => new Promise((r) => setTimeout(r, ms));
let inUse = 0, peak = 0;
await Promise.all(Array.from({ length: 20 }, (_, n) => pool.use(async (conn) => {
peak = Math.max(peak, ++inUse);
conn.tenant = `tenant-${n}`;
await sleep(5);
inUse--;
})));
assert.ok(peak <= 3 && created <= 3, `peak ${peak}, created ${created}`);
await pool.use(async (conn) => { assert.equal(conn.tenant, undefined); }); // reset
await assert.rejects(pool.use(async () => { throw new Error("query failed"); })); // discarded
assert.equal(destroyed, 1);
const held = await Promise.all([pool.acquire(), pool.acquire(), pool.acquire()]);
await assert.rejects(pool.acquire(30), PoolExhaustedError); // exhaustion
held.forEach((c) => pool.release(c));
assert.throws(() => pool.release(held[0]), /not borrowed/); // double return
pool.close();
console.log(`peak ${peak}, created ${created}, destroyed ${destroyed}`);
Implementation notes that interviewers probe:
- The semaphore is acquired before creating an object, and released if creation fails. Otherwise a failing
create()(say the database is down) would permanently shrink the pool by one slot each time. - Track leased objects by identity, not by
equals. Two connections that compare equal are still two sockets. Java uses anIdentityHashMap-backed set, Python keys byid(), and JavaScript'sSetalready compares objects by reference. - Return exactly once. The Java lease uses
compareAndSet, so a secondclose()is a no-op. Returning an object twice would put it in the idle list twice, and two borrowers would then share one connection, which is one of the nastiest pool bugs. - Discard on error. The Python and TypeScript versions treat an exception during use as "possibly broken": the connection might be mid-transaction or in an unknown protocol state. Destroying it is cheap insurance.
- Fairness. The Java semaphore is created fair, so long-waiting borrowers aren't starved by new arrivals. Fairness costs some throughput, and production pools make different choices here.
The borrow/return protocol, in detail
A pool is only as safe as its protocol. Every resource pool needs a clear answer to each of these:
| Question | Good default |
|---|---|
| What happens when all objects are in use? | Wait up to a timeout, then fail with a specific error. Never wait forever. |
| Who returns the object? | A lease or wrapper whose close/dispose returns it, used inside try-with-resources, with or try/finally |
| What state is cleared on return? | Everything a borrower can change: open transactions, session variables, auto-commit mode, isolation level, temporary tables, ThreadLocals |
| How is a broken object detected? | Validate on borrow (cheap check), discard on error during use, and recycle after a maximum lifetime |
| What if the pool is closed while objects are out? | Refuse new borrows, destroy idle objects now and leased ones on return |
Validation
Resources go stale while they sit idle. A database may close idle connections, a firewall or load balancer may silently drop long-idle TCP flows, and a server restart kills them all. Common strategies, which production pools combine:
- Test on borrow: check before handing out, as above. JDBC 4 offers
Connection.isValid(timeoutSeconds), which drivers usually implement as a lightweight ping. It's reliable, but adds a little latency to every borrow, so pools often skip the check for objects used very recently. - Background validation and idle eviction: a housekeeping thread tests idle objects and closes ones that have been idle too long, keeping the pool between a minimum and maximum size.
- Maximum lifetime: retire every object after a fixed age, whether or not it looks healthy, and set that age shorter than any timeout imposed by the database or network. HikariCP's
maxLifetimesetting does this.
Resetting state between borrowers
State bleeding between borrowers is a correctness and security bug, not just a performance one. Examples that have bitten real systems:
- a connection returned mid-transaction, so the next borrower's statements join someone else's transaction;
- a session setting such as
SET search_pathor a tenant ID set for one customer and inherited by the next request; - a
ThreadLocalholding the current user, left set on a pooled thread, so the next task on that thread runs as the wrong user.
The fix is a reset step on return that restores a known baseline (roll back, restore auto-commit and isolation level, clear session variables) and a rule that objects whose reset fails are destroyed. In thread pools, clear ThreadLocals in a finally block, or avoid them for request data.
Sizing a pool
Too small and requests queue for a connection. Too large and you overload the resource the pool was meant to protect. Bigger is not safer.
Start from Little's law. The average number of objects in use equals the arrival rate times the time each one is held:
If a service handles 400 queries per second and each holds a connection for 10 ms, on average 400 × 0.010 = 4 connections are busy. Add headroom for bursts and variance, perhaps two or three times the average, and you're at around 10, far smaller than most people guess.
Then respect the downstream limit across all instances. A pool of 20 per instance across 15 instances is 300 connections. If the database allows 100, the pools will collectively fail at peak, however well each is tuned. This is why large deployments put a server-side pooler (such as PgBouncer for PostgreSQL) in front of the database. See connection pooling for the database side.
Database guidance. The HikariCP project's "About Pool Sizing" article quotes a PostgreSQL-community starting point of about (cores × 2) + effective spindle count connections for the database server, which comes out surprisingly small. The reason is that a database server can only run as many queries at once as it has cores (and disks). Extra connections just add contention and context switching. Treat the formula as a starting point and measure.
Thread pools. For CPU-bound work, about one thread per core. For work that waits on I/O, more. Brian Goetz's Java Concurrency in Practice gives threads ≈ cores × target utilisation × (1 + wait time / compute time). A task that waits 90 ms for every 10 ms of computing can keep about 10 threads per core busy.
Watch for pool-induced deadlock. If a task holding a connection needs a second connection from the same pool to finish (a nested query, a separate audit-log write), then with N concurrent tasks each holding one connection and waiting for another, nobody can proceed. The same happens with thread pools when a task submits a subtask to its own bounded pool and waits for it. Prevent it by never borrowing twice from one pool in one unit of work, by passing the borrowed object down, and by always using borrow timeouts so a deadlock becomes a visible error instead of a hang.
Leaks: the failure mode of every pool
A leak is a borrow without a matching return. Usually it's an exception path that skips the release, sometimes an object stashed in a field and forgotten. Each leak permanently removes one object from the pool. Eventually every borrower times out, and the service looks hung while CPU sits idle. Leaks are hard to spot because the code that leaked has long since finished. The symptom shows up in unrelated requests.
Prevent them structurally:
- Return through a construct that can't be skipped: Java
try (var lease = pool.borrow(...)), Pythonwith pool.lease() as conn:, C#using, TypeScriptpool.use(async conn => ...)withtry/finallyinside. Don't expose bareborrowandreleaseto application code if you can avoid it. - Make the returned handle a proxy whose
close()returns the object to the pool. This is how JDBC pools work: theConnectionyou get from aDataSourceis a wrapper, and callingclose()on it returns the physical connection to the pool instead of closing it.
Detect the rest:
- Record when (and optionally where, with a stack trace) each object was borrowed, and log a warning when one has been out longer than a threshold. HikariCP's
leakDetectionThresholddoes this. - Export metrics for active, idle and waiting counts and for borrow wait time. A rising "active" count with flat traffic is a leak signature.
Why pooling ordinary objects is usually pointless
On the JVM, .NET, V8 and modern Go, allocation is cheap and short-lived garbage is cheaper than intuition suggests:
- Allocation is a pointer bump. Generational collectors give each thread a private allocation buffer (a TLAB on the JVM), so creating a small object is usually a few instructions with no locking.
- Dead young objects cost almost nothing to collect. A young-generation collection copies out the live objects and reclaims the rest wholesale, so its cost tracks what survives, not what was allocated. Objects that die young are freed essentially for free.
- Pooled objects live forever, which is worse for the collector. They get promoted to the old generation, where they're scanned during major collections. When they're updated to point at new young objects, they create old-to-young references that the collector must track through write barriers and card tables. A pool can increase GC work.
- The JIT may not allocate at all. HotSpot's escape analysis can replace a non-escaping object with plain local values (scalar replacement), so the allocation disappears entirely. Pooling defeats that optimisation, because a pooled object escapes by definition.
- Pools add their own costs: synchronisation on borrow and return (a contention point under load), reset logic that must be kept in step with every field, and the risk of leaks and state bleeding. All of that buys nothing if creation was cheap.
In CPython, objects are freed by reference counting as soon as the last reference goes, and the interpreter already keeps internal free lists for common types such as floats, tuples, lists and dicts. Pooling small Python objects by hand adds overhead and rarely gains anything.
When pooling ordinary memory is still justified:
| Situation | Example |
|---|---|
| The object wraps a native or OS resource | Sockets, file handles, GPU buffers, threads |
| The object is very large | Big byte arrays or direct buffers. Netty pools buffers with PooledByteBufAllocator, and .NET has ArrayPool<T>.Shared |
| Initialisation, not allocation, is the cost | Compiled regexes, parsers with loaded grammars, cipher contexts. These are often better cached and shared if they're thread-safe, and pooled only if they're not |
| Hard latency limits where any GC pause matters | Game loops (pooling bullets and particles in Unity is a well-known practice), low-latency trading, audio processing |
| A runtime with a weak or non-generational collector | Some embedded and older mobile environments |
The rule: measure before pooling plain objects. Use an allocation profiler or GC logs to find a real problem first. Usually there isn't one.
Don't pool virtual threads
Java 21's virtual threads (JEP 444) are designed to be cheap enough to create one per task. The JEP explicitly advises against pooling them. If you need to limit concurrency, for example to protect a database, use a Semaphore or the connection pool itself, not a fixed-size pool of virtual threads.
Real-world uses
Java
- HikariCP, Spring Boot's default JDBC connection pool, with settings including
maximumPoolSize,connectionTimeout,maxLifetime,idleTimeoutandleakDetectionThreshold. - Apache Commons Pool (
GenericObjectPool), a general-purpose pool used under the hood by Apache DBCP2 and by Jedis'sJedisPoolfor Redis connections. Its factory interface (create, validate, activate, passivate, destroy) is close to the design above. ThreadPoolExecutor, plus theExecutorsfactory methods andForkJoinPool.commonPool(), which pool threads and queue tasks.- Netty's
PooledByteBufAllocator, which pools large network buffers, one of the justified cases for pooling memory.
.NET
- ADO.NET providers such as SqlClient pool connections by default:
SqlConnection.Close()returns the physical connection to a pool keyed by connection string. ThreadPool,ArrayPool<T>.Shared, andObjectPool<T>inMicrosoft.Extensions.ObjectPool.IHttpClientFactorypools and recycles the underlying HTTP message handlers, avoiding both socket exhaustion and stale DNS.
Python
concurrent.futures.ThreadPoolExecutorandProcessPoolExecutor, andmultiprocessing.Pool.- SQLAlchemy's
QueuePool(its default pool for most databases, configured withpool_size,max_overflow,pool_timeout,pool_recycleandpool_pre_ping) andpsycopg_poolfor psycopg 3.
Node
pg.Poolin node-postgres, with options such asmax,idleTimeoutMillisandconnectionTimeoutMillis, and apool.query()shortcut that borrows and returns for you.- libuv's worker thread pool (size set by
UV_THREADPOOL_SIZE, default 4) runs file system calls, DNS lookups and some crypto off the main thread. http.Agentkeeps sockets open per host for reuse when keep-alive is enabled.
When NOT to use it, and the costs
- Creation is cheap. Ordinary objects, value objects, DTOs, small collections: allocate and let the garbage collector do its job.
- The resource has no natural limit and no setup cost. Then a pool adds coordination for nothing.
- A library already pools it. Use the driver's or framework's pool. Writing your own connection pool for production is rarely a good idea, because the established ones have years of fixes for edge cases you haven't hit yet.
- The object is stateful in ways that are hard to reset. If you can't reliably restore a clean state, reuse is a security risk. Create fresh objects instead.
- The object is thread-safe and stateless. Share one instance (a cache, or a single injected instance). Pooling is for objects that must be used exclusively by one borrower at a time.
Costs:
| Cost | Why it matters |
|---|---|
| Leaks | A missed return permanently shrinks the pool, and exhaustion looks like a hang |
| State bleeding | Unreset state crosses requests, users or tenants |
| Stale objects | Idle resources die silently, so validation and lifetimes are needed |
| Contention | Every borrow and return synchronises on shared structures |
| Tuning | Size, timeouts and lifetimes must match the workload and the downstream limits |
| Deadlock risk | Nested borrows from one pool can deadlock under load |
Related patterns and how they differ
- Flyweight. Both reuse objects, but Flyweight shares immutable objects among many users at the same time, and Object Pool lends mutable objects to one user at a time. Flyweights never need returning. Pooled objects always do.
- Cache. A cache keeps results by key and shares them concurrently. A pool keeps interchangeable resources and lends them exclusively. A pool of compiled-regex objects is a pool. A map from pattern string to compiled regex is a cache.
- Singleton. A pool is usually a single shared instance per resource per process. Inject it rather than making it a classic singleton, so tests can use a small pool or a fake.
- Factory Method. The pool delegates creation to a factory (the
PooledObjectFactoryabove), and a pool can itself be seen as a factory that sometimes returns a recycled object instead of a new one. - Proxy. Pooled connections are typically handed out as proxies whose
close()returns the real connection to the pool, which is what makestry-with-resources andwithwork naturally with pools. - Prototype. A pool could create new members by cloning a configured prototype, but resources such as connections can rarely be copied, so pools almost always use a factory.
The smell that suggests it
- Creation dominates latency. Profiling shows connection setup, TLS handshakes or thread creation inside the request path.
- "Too many connections" errors, or file-descriptor exhaustion, under load.
- Unbounded concurrency against a limited resource:
new Thread(...)per request, a connection per message, unlimited parallel calls to a rate-limited API. - Code that caches one expensive non-thread-safe object per thread, and runs into trouble when the thread count grows.
The smell that suggests you should remove a pool:
- pools of small plain objects with no external resource, added "for performance" without a measurement;
- reset methods that must be updated every time a field is added, and occasionally aren't.
Interview questions
- "Design a database connection pool." Clarify the requirements first: maximum size, behaviour when exhausted (wait with timeout), validation, idle eviction, maximum lifetime, thread safety, leak detection and metrics. Then present the structure above: a semaphore for capacity, an idle deque (LIFO), a leased set tracked by identity, a factory for lifecycle, and leases or proxies that return on
close(). Cover failure paths: creation failure releases the permit, broken connections are destroyed, double returns are ignored or rejected. - "How do you size a pool?" Little's law: average in use = throughput × hold time, plus headroom for bursts. Then check the total across all instances against the downstream limit. For a database, more connections than the server can run concurrently only adds contention. Measure wait time and utilisation, and adjust.
- "What happens when the pool is exhausted?" Borrowers wait up to a timeout and then fail with a clear error. Alert on wait time and timeouts. Never wait forever, because that turns a slow dependency into a full outage.
- "How do you prevent and detect leaks?" Prevent: return through
try-with-resources,withorfinally, ideally with a proxy whoseclose()returns. Detect: record borrow time and stack trace, warn when an object has been out longer than a threshold, and graph active versus idle counts. - "Why not just pool every object to reduce garbage collection?" Modern generational collectors make short-lived objects very cheap to allocate and collect. Pooled objects live long, get promoted, add old-to-young references and defeat escape analysis. Pools add contention and reset bugs. Pool only resources or very large buffers, and only after measuring.
- "How is a connection pool different from a cache?" A pool lends interchangeable objects exclusively and expects them back. A cache shares values by key concurrently and doesn't expect anything back.
- "How can a pool deadlock?" A unit of work holding one object waits to borrow a second from the same exhausted pool, while all other holders do the same. Avoid nested borrows from one pool, pass the borrowed object down, and use borrow timeouts so it fails visibly.
- "What should happen to a connection that threw an exception mid-transaction?" Treat it as suspect: roll back if possible, or destroy it and let the pool create a fresh one. Never return a connection in an unknown state to the idle list.
- "Should you pool Java virtual threads?" No. They're designed to be created per task, and the JEP advises against pooling them. Limit concurrency with a semaphore or with the resource's own pool.
Further reading
- Connection Pooling: the database side, including server-side poolers.
- Mutexes, Semaphores and Monitors: the primitives a pool is built from.
- Classic Synchronization Problems: bounded buffers and producer-consumer, the pool's close relatives.
- Creational Patterns: the grouped overview of the GoF creational patterns.
- Design Patterns: One Page Each: the index of every pattern page.