The Server-First Reader Pool: Exact Connection Reuse and Hot-Path Performance

Once a proxy decides which replica should serve a read, it still has to find or create a connection to that replica — fast. Under high concurrency, that step can easily become the bottleneck: every query scans connection lists, updates shared counters, and contends on locks. This article is about how Polar Proxy organizes its reader connection pool and trims the hot path so that connection handling stops competing with the actual work of answering queries.

Part I — Background

What a connection pool does

Opening a PostgreSQL connection is expensive: a TCP handshake, TLS negotiation, authentication, and startup parameter exchange. A proxy therefore keeps a connection pool: a set of already-open backend connections it reuses across client requests. When a request needs a backend, the proxy takes a compatible idle connection from the pool; when the request is done, it returns the connection to the pool.

Compatibility is a challenge. Two connections are interchangeable only if they were opened with the same user, the same database, the same session options, and — in our case — the same PolarDB startup profile (whether the connection negotiates LSN reporting and in-band waits). A connection that negotiates those extensions also records a client startup identity — today the endpoint, with the TLS fields kept as reserved placeholders in the same key — and that identity joins the reuse match. A mismatch means the backend would behave differently for the client, so a proxy must match on all of these.

Choosing a replica: power-of-two-choices

When several replicas could serve a read, the proxy must pick one. The naive approaches — round-robin or “least connections” — each have problems (round-robin ignores load; least-connections is expensive to track exactly). A popular compromise is power-of-two-choices (P2C): pick two servers at random, in proportion to their configured weights, and among them choose the one with the lower relative load — active connections normalised by weight. The LSN target is not part of that comparison. It is decided separately: a read with a wait target prefers the readers that have already replayed far enough (so it can skip the server-side wait), and falls back to the rest — which then wait server-side — only when none has, or when the ready ones cannot hand over a connection. P2C gives most of the load-balancing benefit of scanning every server at a fraction of the cost.

Where the time actually goes

A database proxy is an event-driven program: a fixed number of worker threads each run an event loop that moves many client and backend connections forward. In such a design, three kinds of cost dominate: lock contention (two threads touching the same shared state), allocation churn (temporary objects created and freed per request), and data copying (moving bytes between buffers).

Part II — What upstream ProxySQL does, and why it slows down

Upstream ProxySQL’s generic connection pool works, but it was not built for LSN- and transaction-aware routing.

No exact matching. Its compatibility design is rough. For PolarDB reads, the proxy must reuse a connection whose startup profile, LSN tracking, and transaction metadata match the request exactly. The generic pool cannot express those dimensions — to its checks, a backend that lacks the required PolarDB state is indistinguishable from one that has it. A reuse policy built on it must choose between risking the routing guarantees and refusing reuse whenever the state it cannot see might differ, creating connections needlessly.

Repeated work per query. On every read, the upstream path re-scans connection lists, re-checks server status, rebuilds a temporary candidate array, and re-holds the hostgroup-manager lock. Much of that work was already done moments ago for the previous query, yet the same scans, checks, and lock acquisitions run again.

Shared-lock contention on unrelated hot spots. For example, query-digest statistics — the per-query counters used for monitoring — are updated in a shared map on every query behind one process-wide lock, so under load every worker contends on the same lock word. And the logger’s flush is checked on every event-loop pass even when logging is disabled, taking locks for a no-op.

Per-packet copying. When a backend returns a large result, upstream forwards it to the client by copying each packet through an intermediate buffer. For a big SELECT, that is a lot of needless memcpy.

Part III — What the Polar Proxy team built

One owner for reader connections

The core connection pool acts as the single owner of every reusable reader connection. The reader pool no longer keeps its own separate connections; it only decides which server to use and which policy to apply (LSN target, lag cap, warmup). When a connection is in the pool, there is exactly one place it lives and one rule for how it moves in and out. Even a connection a worker is briefly holding for local reuse stays accounted in the core pool’s USED list — the worker cache is a reuse handle, not a second owner.

Each server’s connection list now carries a four-part exact-match key — startup profile generation, user/database hash, client startup identity, and startup options — plus an exact-match index. Lookup finds a connection by that key, so the proxy reuses a connection only when it is truly available. The identity slot is compared only for connections whose startup profile carries a client identity; a profile that sends no startup parameters leaves the slot zero, which means “skip this check”. Connections move between FREE and USED under a single per-server mutex, with the list, the index, and the counters updated together.

Reuse locally, choose with P2C

Two micro-optimizations cut the per-query cost of finding a connection:

• Worker-local reuse. Once a worker has selected a server, it keeps the matching connection it just used in a thread-local cache — by default, only for the remainder of that event-loop pass. The next read on the same server reuses it without touching the shared pool at all. A connection that no longer matches — the server went offline, the client identity changed, or the pool state is no longer exact — is destroyed rather than pooled; one that still matches is kept locally unless another worker is already waiting for that match key, in which case it goes to the shared pool.
• P2C selection. When more than two readers compete, the proxy picks one by comparing two candidates’ relative load, and only those two have their shared load counters read. With exactly two configured readers there is nothing to sample: a worker-local weighted sequence alternates between them — strictly when the weights are equal, in the configured ratio when they are not — and a target-bearing read shifts the choice to whichever reader has already reached the LSN target. With three or more equal-weight readers and an unconstrained request — no replication-lag cap, no excluded endpoint, free to create a connection — it first samples a random pair, and it stops there — choosing the less-loaded one — only when both are usable and, for a target-bearing read, both have already reached the LSN target. Otherwise it expands to the full list, reusing the already-evaluated pair instead of re-evaluating them.

Wait for capacity instead of spinning

When every suitable connection is in use and no creation slot is free, upstream ProxySQL spins and re-scans. The Polar Proxy instead records a pending request — its match key, reader plan (replica requirement and lag cap), and wait target — and parks it: the event loop keeps serving other sessions while the request waits for a targeted wake or a bounded retry deadline. When a connection is returned or newly created, it is handed directly to that pending request, and only the assigned worker is woken — a byte on its poll wakeup pipe brings that event loop around early. The wait starts only after capacity pressure is confirmed, so the common first-hit case takes the fast path unchanged. Background warmup — pre-opening connections so that transaction-split reads do not stall on a cold reader — runs in a dedicated background thread, opens sockets outside the hostgroup-manager lock, and adds available connections straight to the FREE list.

Take the locks off the hot path

Three changes remove shared-state contention:

• Worker-local query digests. Instead of updating the shared digest map per query, each worker accumulates digest statistics in its own small map and flushes it to the shared store when the map hits its entry limit, when the worker runs its periodic housekeeping, or when someone reads the counters. The shared map becomes the final reporting store, not the per-query contention point.
• Skip the logger when it is off. The worker first reads the two “logfile open” flags. If both the event and audit logs are closed, it returns immediately — no clock read, no shared context lookup, no buffer lock. The scheduled flush and rotation still run when logging is actually enabled.
• Batch parent-counter updates. Each backend connection accumulates its own byte and query counts and folds them into its server’s shared counters in batches — every 64 queries, or once either direction’s pending byte count crosses 64 KiB — rather than one atomic increment at a time.

Stop copying result bytes

Finally, the response path avoids needless copying in two ways:

• The patched libpq can identify and detach contiguous row packets from a result (PSpeekRowRun / PSdetachRowRun). This fast-forward path (pgsql-polardb_result_fast_forward) is opt-in and off by default; when enabled, the proxy transfers a whole run of rows to the client by moving ownership of the buffer, not by copying each packet.
• Once a batch has accumulated in the output queue — at least two packets, or one too large for the classic buffer — and the frontend is a plain socket with no TLS and no partial packet in flight, the proxy builds an iovec directly from the queued result packets and sends them with a single scatter-gather sendmsg call: at most 64 packets within a byte budget per call, with the remainder following on subsequent writes. Single small packets stay on the classic buffered path. The direct path is enabled by default.

### The result

The combined effect is that connection handling does not add extra overhead per query. Reader connections are matched exactly and reused without cross-thread contention; on the common path where pair sampling applies, selection is O(1)-ish via P2C plus worker-local reuse; capacity pressure leads to waiting, not spinning; and the two unrelated hot spots — digest accounting and logger flushing — are taken off the per-query path. Result forwarding avoids copying for the common large-result case.

None of this changes the consistency guarantees. It is a pure performance layer: the same LSN-based routing and the same safe-fallback rules, running on a pool that no longer gets in the way. The measure of success is simple — under high concurrency, the proxy spends its cycles routing and forwarding queries, not fighting over its own bookkeeping.