etcd Write-Ahead Log: How a Raft Entry Reaches Disk and Replays on Restart

Raft can only call an entry committed once it is durable on the disks of a quorum, and etcd's durability primitive is the write-ahead log. The WAL is an append-only sequence of length-prefixed, CRC-protected records on disk: before any raft entry is treated as committed and before the apply loop is allowed to execute it, that entry is encoded into a record, appended, and (when the entry demands it) flushed to stable storage with an fsync. The same file format is what lets a node that lost power come back up, replay its log, and reconstruct the exact raft state it had before the crash. WAL is the struct that owns this: an open file directory, an encoder for the tail segment, a running CRC, and the index of the last entry it has seen.

This spoke follows a single raft entry from the moment the ready loop hands it over to the moment it is safely fsynced, then follows the reverse path a restarting server takes to replay it. The analysis is generated from Symvanta's code graph of etcd pinned at commit 6006f40. As with the write path itself, the WAL is driven through interface and channel boundaries rather than named calls, so its central methods (Save, sync, ReadAll) show as zero-caller nodes in the graph even though one of them runs on effectively every write.

The moving parts

• WAL
• WAL.Save
• encoder.encode
• WAL.sync
• WAL.cut
• NewDecoderAdvanced
• decoder.decodeRecord
• Record.Validate
• WAL.ReadAll

The write side and the read side are near mirror images. On the way to disk, WAL.Save is the entry point, encoder.encode turns a walpb.Record into framed, CRC-tagged bytes, and WAL.sync is the fsync that makes those bytes survive a crash. On the way back, NewDecoderAdvanced constructs the reader over the on-disk segments, decoder.decodeRecord pulls one record at a time, and Record.Validate checks each record's stored CRC against the running checksum so a torn or corrupted tail is caught rather than silently replayed. WAL.cut is the housekeeping that keeps segments bounded, and NewDecoder is the strict wrapper over NewDecoderAdvanced that refuses to continue past a CRC mismatch, the entry point both a replaying server and the offline etcdutl inspection tools decode through.

How it works

Here is what happens to a raft entry between the ready loop calling Save and the same entry being replayed after a restart.

1. WAL.Save receives the raft HardState and the slice of entries the ready loop just pulled off the Raft library. It short-circuits when there is nothing to write, then computes mustSync from raft.MustSync, which decides whether this batch requires a durable flush or can ride along in the OS page cache.
2. WAL.saveEntry runs once per entry. It marshals the raft entry into a walpb.Record tagged as an entry record and hands it to the encoder; Save then writes the hard state as a state record after the entries.
3. encoder.encode frames the record. It folds the record's payload into the running CRC (seeded when the encoder is built by newEncoder), stamps that checksum into the record, writes a length prefix, and pads each record to an 8-byte boundary so the decoder can detect a partial trailing write. The CRC chaining is what ties the integrity of each record to every record before it.
4. WAL.sync is the durability barrier. When mustSync is set and the current segment still has room, Save calls sync, which invokes fileutil.Fdatasync to force the appended bytes to stable storage and records how long the flush took. Only after this call returns is the entry genuinely durable, which is the precondition Raft needs before it treats the entry as committed.
5. WAL.cut rotates the log. When Save sees the tail segment has grown past SegmentSizeBytes, it closes the current file, opens a preallocated next segment, and starts it with a fresh CRC record via WAL.saveCrc so the chain continues cleanly into the new file. Bounded segments are what make compaction and lock release cheap later.
6. Snapshotter.SaveSnap is the compaction half. Periodically the server writes a full snapshot of applied state through the snap package, and WAL.SaveSnapshot records a small snapshot marker in the log itself; WAL.ReleaseLockTo then unlocks and lets old segments below that snapshot be reclaimed, so the log does not grow without bound.
7. WAL.ReadAll is the restart path. A recovering server opens the WAL at its most recent snapshot with Open, and ReadAll walks the segments from that point forward, dispatching on each record's type to rebuild the metadata, the last HardState, and the slice of entries the node had persisted.
8. decoder.decodeRecord does the low-level reading under ReadAll. It reads a length, reads the framed record, and calls Record.Validate to verify the stored CRC against the running checksum built with crc.New. A mismatch at the very end of the log is the expected signature of a crash mid-append and is tolerated as a truncated tail; a mismatch anywhere else is real corruption and stops recovery. Once replay finishes, the entries feed straight back into the same apply loop that produced them, and the node is exactly where it left off.

Where it connects

The WAL sits directly under the ready loop that the sibling spoke on Raft consensus describes: raftNode.start is what calls Save on every batch of committed entries, and the durability that Save provides is the precondition that makes the whole "committed means safe" contract in that spoke true. In the other direction, the WAL is consumed once per process lifetime, at startup, by the recovery code that calls ReadAll before the server begins serving clients.

On the etcd module map, the decode side of this subsystem is its own cluster: Data Parsing Utilities, whose hub is NewDecoderAdvanced, is the WAL and snapshot decoding layer that both the running server and the offline etcdutl tools read through, which is why the strict NewDecoder and the lenient NewDecoderAdvanced are called out as load-bearing symbols on the hub page. The snapshot machinery that compaction depends on clusters with the mvcc and peer-URL data structures in Data Structure Management, tying the log's growth control back to the storage layer whose size it is trying to bound.

By the numbers

The write path is dominated by one method: WAL.Save is 43 lines (wal.go 958 to 1000) and everything else on the hot path (saveEntry, encode, sync) is small and called from it. The read path is lopsided the other way: ReadAll is a 124-line function (wal.go 471 to 594) because it has to handle every record type, snapshot matching, and the torn-tail case, while decodeRecord beneath it is 68 lines. The two decoder constructors make the integrity policy explicit in the type system: NewDecoderAdvanced takes a continueOnCrcError flag, and NewDecoder is the one-line wrapper that hard-codes it to false, so the default everywhere in the running server is to stop on corruption rather than guess.