Zero-Copy I/O for the Shared Storage Daemon: Removing the Last Copy from the WAL Path

Every commit in PolarDB ends the same way: the WAL record must reach shared storage. On the way there, the bytes move from a WAL buffer, through the shared-storage daemon, to a device. In the default path, the daemon copies the caller’s bytes into a pool it owns, and then writes the device from that pool. This copy is the last one left on the commit-critical path, and it provides no benefit. This article is about the change that removed it: zero-copy I/O. With zero-copy, the daemon reads and writes PolarDB’s own shared-memory buffers in place.

Part I — Background

How a page gets to shared storage

PolarDB’s storage is handled by a dedicated file-system daemon, pfsd. pfsd performs the device I/O for the cluster’s data files and WAL. When a backend writes a buffer, it passes the bytes to pfsd. pfsd then performs the device write.

There are two kinds of data movement between pfsd and postgres. The first is the control path. The backend and pfsd run in different processes, so they coordinate through shared memory and futex wakeups. The second is the data path: the bytes that must reach the device. The control path is not free — waking the pfsd worker thread goes through a futex, which adds scheduling latency — but it moves only a few words of coordination. The data path is where the bytes move. It is also where optimization yields the most benefit.

In the default pfsd write path, pfsd does not write the caller’s buffer directly. It copies the payload into a pool of memory that pfsd itself owns, then writes the device from that pool. The copy exists because pfsd cannot see a caller’s buffer at all unless that buffer already lives in shared memory — and most buffers do not. The cost is that every WAL flush incurs one extra memory copy. WAL flush is the most latency-sensitive write in the database.

Why the copy was there

The copy exists for one reason, and that reason will never fully go away: pfsd and postgres are separate processes, and a process can only read memory it can see. A caller may want to do I/O on any buffer — one returned by malloc, one allocated on the stack, or one the linker placed in the data segment. None of those buffers can be made visible to both pfsd and postgres at the same address, so pfsd cannot read them directly. The caller must copy the bytes into a region pfsd can see: a pool pfsd itself owns.

For the commit path, which already pays for a memcpy into the WAL buffers and the system-call overhead of the flush, that copy provides no benefit. But it is unavoidable for arbitrary buffers. What zero-copy targets is the subset of buffers that already live in shared memory — the ones where the copy is avoidable.

Part II — The problem

The copy is on the commit-critical path. A backend assembles a WAL record and waits for it to become durable. In the default path, it must wait while pfsd copies those bytes a second time before the device write can begin. Under write load, that copy adds latency to every commit. It also consumes memory bandwidth that the rest of the system is also using.

The copy also scales badly with WAL volume. A write-heavy PolarDB deployment produces more WAL. More WAL means more bytes for pfsd to copy, and more overhead from the defensive copy.

The goal is precise: remove the copy from the data path for the buffers that do not need it. The copy exists only because ordinary buffers are invisible to pfsd. PolarDB already has a class of buffers that pfsd can see directly — the ones in its own main shared-memory segment. For those, the copy is pure overhead.

Part III — What we built

One memfd, one registered region

The key insight is that PolarDB’s shared memory is already the kind of buffer pfsd can read directly. It is a single region that every postgres backend is already attached to. It is allocated once at startup, exists for the lifetime of the postmaster, and is never freed while the cluster runs. The only thing missing is that pfsd itself is not attached to it. Zero-copy makes pfsd attach to that same region, so the copy for shmem-resident buffers becomes unnecessary.

The change backs the entire main shared-memory segment with a single memfd. The memfd is mapped MAP_SHARED in the postmaster before it forks, and registered with pfsd exactly once. Every buffer in shared memory — the WAL buffers, the shared buffer pool, the copy-buffer pool, SLRU pages, and the checksum-copy scratch — then resides inside that one region. pfsd’s write and read shim (pfsd_pwrite_zc / pfsd_pread_zc) recognises any pointer that lies inside the segment. It performs the I/O directly from that pointer, with no intermediate copy.

Two details make this robust. First, the buffers that need it are aligned to PG_IO_ALIGN_SIZE unconditionally, not only when zero-copy is active. The sizing and the layout of the segment can therefore never disagree. Second, alignment is a fast-path qualifier, not a correctness requirement. If a buffer is misaligned, the shim’s alignment guard falls back to the copying path. Zero-copy is an optimisation that can be skipped.

The result

The result is that the last memory copy is removed from the WAL commit path. A WAL flush now writes directly from the WAL buffers to the device, instead of first being copied into a pfsd-owned pool. The same applies to any shared-memory buffer pfsd accesses — dirty data pages, SLRU pages, the checksum scratch. The saving is therefore not limited to WAL.

When shared storage and pfsd are present, turning on polar_enable_zero_copy removes the copy. Turning it off restores the exact previous behaviour. The correctness guarantee that motivated the copy is preserved: pfsd skips the copy only for buffers it can actually see, and falls back to the copy for everything else.