DataMax: A Dedicated WAL Log-Store Node for Near-Synchronous Durability

Every durable database change in PolarDB passes through one place before it is safe: the write-ahead log. A replica is just a machine that stores that log and replays it. But storing and replaying are two very different costs — and a cluster that wants the durability of a second copy of the WAL may not want to pay for the full replay machinery of a second server. This article is about DataMax, a dedicated node that keeps a durable second copy of the WAL and cascades it onward, without ever replaying a single record. It streams the WAL like a standby, and because it is meant to sit next to the primary, it can carry the synchronous-commit acknowledgement itself — at intra-AZ latency — while the standbys it feeds sit far away.

Part I — Background

WAL is the cluster’s only source of truth

In PolarDB’s shared-storage architecture, one read-write node produces WAL and the read-only nodes consume it to stay consistent. WAL is not merely a crash-recovery mechanism — it is the channel through which every replica learns about new data, and the object every node must agree on. A full replica does two things with that stream: it stores the WAL durably, and it replays it into its own buffer pool so it can serve reads.

Those two jobs have very different prices. Storing WAL is cheap — it is sequential I/O to a log directory. Replaying WAL is expensive — it means parsing every record, modifying pages in memory, resolving conflicts with live queries, and keeping a full copy of the database’s data files on disk. A full replica pays for a complete database it may never be asked to serve.

The idea of a log-only node

What a cluster sometimes wants is a third kind of node: one that participates in the replication chain and holds a durable copy of the WAL, but skips the replay entirely. It offers the same storage benefit as a standby — a second machine where the WAL is safely on disk — at a fraction of the storage and CPU cost, because it never builds or maintains a database. And unlike a distant synchronous standby, it does not make the primary pay a cross-AZ round trip for durability: the DataMax is placed in the same availability zone as the primary and can be named in synchronous_standby_names, so commits wait only for that short hop while the remote standbys replicate from the DataMax asynchronously.

That is DataMax. It streams WAL from an upstream primary, stores it durably, and cascades it to any downstream standbys, all without replaying it. The catch is that a node that runs no transactions cannot derive some of the bookkeeping a replication participant normally computes for itself.

Part II — The problem

A transactionless node has no epoch of its own

Replication depends on more than raw WAL bytes. Two pieces of state are normally produced by a node’s own transaction machinery, and a node that runs no local transactions cannot produce them.

The first is the transaction-id epoch. PostgreSQL’s 32-bit transaction IDs wrap around. To make them unique forever, PostgreSQL pairs a 32-bit xid with a separate 32-bit epoch counter; together they form a 64-bit FullTransactionId. A primary advances that epoch as its xid counter rolls over. A node that never commits anything has no epoch of its own.

The second is the WAL horizon — how far back the WAL stream must be retained, and which segments are safe to remove. On a normal node this is computed from the oldest replication slot and the checkpoint retention window, never from open transactions, which constrain vacuum rather than WAL. A node that runs no checkpoints has no such horizon of its own to speak of.

A DataMax node therefore needs to borrow both. The primary already knows its own next XID, its epoch, and how far back its WAL is retained — so the primary ships that state along with the WAL. The DataMax records it, uses the borrowed epoch to build a sane vacuum horizon for feedback, and forwards the retention state onward to a cascaded DataMax.

A log-only node must still look like a replica

The second problem is bootstrap and handshake. Streaming replication starts with IDENTIFY_SYSTEM, in which the primary reports its system identifier and timeline. A normal standby gets its system identifier from pg_basebackup, which clones the primary’s control file. A DataMax node holds no data and should not need a full base backup — but it must still present the same system identifier as its upstream, or the handshake rejects it. A log-only node therefore needs a way to adopt the upstream’s identity without cloning its data.

Part III — What we built

A node that streams, stores, and cascades — never replays

DataMax is implemented as a dedicated module, polar_datamax, plus a new postmaster state. The node runs a full WAL receiver and a walsender, so it sits naturally in a replication chain: primary → datamax → standby works exactly like primary → standby → standby. The difference is what happens to the bytes. The WAL receiver writes incoming records to a dedicated WAL directory and persists a small metadata header after each flush; the walsender serves cascade WAL straight out of that directory. Nothing ever calls the redo path, because no page is ever replayed.

Because there is no data to flush, DataMax skips the parts of recovery that only exist to bring data files up to date. There is no restart point to write, no buffers to flush, and the archive machinery routes to the DataMax WAL directory rather than a primary’s pg_wal.

Borrowing epoch and horizon through the ‘e’ message

The epoch and horizon problem is solved by extending the WAL stream itself. Alongside the regular 'w' fields that mark WAL positions, a new 'e' message carries four extra fields: the primary’s nextXid, its epoch, its last valid LSN, and its last removed segment. The DataMax’s WAL receiver records that state and maintains a list of valid received LSNs. From there the state flows in two separate directions.

Downstream, the DataMax forwards the borrowed epoch and the retention state through its own walsender — but only to a downstream that is itself a DataMax. The 'e' message is chosen from the mode the downstream asked for in START_REPLICATION, so a plain cascaded standby connects as a standby and receives ordinary 'w' messages; it derives its own epoch and its own WAL horizon from its own slots. Upstream, the DataMax sends hot-standby feedback built from the borrowed epoch and from the xmin of its own replication slots. It has no local transactions, so it has no xmin of its own; if it let its frozen-xid state speak, it would advertise a vacuum horizon older than the cascaded standby’s slot xmin and wrongly hold up vacuum on the primary. Feeding back only the slot xmin — which it merely records and forwards — avoids that. The DataMax’s own walsender then validates a cascaded standby’s feedback xmin against the epoch the primary shipped, and the primary validates the DataMax’s feedback the same way, against its live epoch. The last-removed-segment field keeps the DataMax from ever removing WAL the upstream still needs.

This is the mechanism that lets a transactionless node behave as a full participant in the replication chain: every piece of state it needs, it receives in-band and reflects back out.

Identity without a base backup

The bootstrap problem is solved by teaching initdb to accept an explicit system identifier. A DataMax node is initialised with the upstream’s identifier, so when it connects and asks IDENTIFY_SYSTEM, it presents the identity the primary expects — no pg_basebackup required. On a fresh start it requests streaming with a deliberately invalid timeline and LSN, adopts the primary’s timeline after the handshake, and the primary seeds its slot’s restart_lsn and streams from its smallest retained WAL. The node walks itself into the replication chain from a cold, empty state.

Cascading, timeline switches, and promote

Three more pieces round out the feature. Timeline history is fetched and served against the DataMax WAL directory, so the node follows an upstream timeline switch and serves older timelines downstream without a full data copy. And on promote, a DataMax node can gate promotion on having received all upstream WAL: it exchanges a 'p' request and an 'l' reply with the upstream, and the reply carries the LSN it must have received before it may declare itself promoted. That wait is off by default (polar_enable_promote_wait_for_walreceive_done = off); the operator can force promotion past it with pg_ctl promote -f. When enabled, it is what makes a DataMax a safe failover target: it never promotes past WAL it has not yet durably stored.

The result

The outcome is a new node type for the shared-storage cluster: one that provides a durable second copy of the WAL — synchronous at intra-AZ latency when it is named as the commit’s sync partner — and can serve as a cascade hub, at the storage and CPU cost of a log directory rather than a full database. It participates fully in replication — epoch, horizon, timeline, and promote — by borrowing state in-band instead of deriving it from local transactions.

The follow-up work was the usual hardening: rejecting a timeline-0 handshake from an upstream that had not yet established its own timeline (which could otherwise crash the node), fixing a use-after-free in the valid-LSN list on exit, dropping a bogus promote assertion, and terminating WAL file names correctly in the backup-import tool. For a DBA the takeaway is a new knob — polar_datamax_mode — and a new answer to the question “how do I get a second durable copy of the WAL without paying for a whole second server?”