Skip to content

Durability & Backup

The delta log is the WAL

There is no separate journal: the row-delta buffer itself is an on-disk append-only log. A write is framed (length | CRC32-C | payload), appended, fsync'ed — and only then acked.

Group commit: appends arriving within the -group-commit window (default 50ms) share one write+fsync. 0 means fsync on every append. Durability is never optional; the window only tunes how it is amortized.

Crash recovery

On open, the engine replays every surviving log file into a fresh memtable. A torn or corrupt tail (the write that was in flight when power died) is detected by CRC and truncated away — everything acked is intact, by construction.

If the crash landed between a merge finishing its segments and deleting its logs, the replayed rows already live in a segment. The next merge deduplicates exact duplicates, so re-merging is idempotent.

CHECKPOINT and backups

Copying a live database directory mid-write is unsupported. The contract is:

CHECKPOINT;

which forces the delta into columnar, seals the segments and drops a CHECKPOINT marker file. After it returns, a plain file copy is a consistent snapshot:

psql "..." -c 'CHECKPOINT;' && rsync -a /var/lib/xcon-db/dbs/firm1/ backup:/firm1/

(Also available as POST /checkpoint over HTTP.) Even without a checkpoint, a restored copy self-heals through log replay — the checkpoint just guarantees the copy is minimal and merge-complete.

Replication is deliberately outside the core: backup + re-route the firm to another node.