etcd Raft Consensus: Put to Committed Write

etcd's promise is that a successful write is durable and linearizable: once a client's Put returns, every replica in the cluster agrees the key changed, and no later read is allowed to miss it. That guarantee comes from the path a write takes through Raft before it is ever permitted to touch storage. EtcdServer.Put writes nothing. It wraps the client's PutRequest in an InternalRaftRequest, proposes it to the local Raft node, and blocks until the cluster has committed that entry and the apply loop has executed it. The key only changes inside applierV3backend.Put, which runs later, on a different goroutine, after consensus is already reached.

This spoke traces that whole path, from the gRPC handler to the applied mutation, as far as the static call graph follows it. The analysis is generated from Symvanta's code graph of etcd pinned at commit c34dc7e, and one artifact of the graph is worth stating up front: the two halves of the write path show up as two disconnected runs. The propose side reaches from the gRPC handler down to the Raft proposal, the apply side reaches from the server's run loop down to the applier, and the hop between them is a Go channel handoff, so no call edge joins them. The consensus algorithm itself lives in an external module, go.etcd.io/raft/v3 (pinned at v3.7.0), which etcd's server drives through a ready loop.

The moving parts

• EtcdServer.Put
• EtcdServer.processInternalRaftRequestOnce
• exceedsRequestLimit
• raftNode.start
• EtcdServer.apply
• EtcdServer.applyEntryNormal
• applierV3backend.Put
• storeTxnWrite.Put
• Read.LinearizableReadLoop

The two ends of the write path are the pair to hold in your head. EtcdServer.Put is a thin 14-line handler: build the internal request, propose, wait, return. applierV3backend.Put is the function that finally writes the key, and it is three lines long: one call straight into the mvcc transaction package. The graph records no inbound call edge into it from the apply loop, which is what a handler dispatched through an interface off a committed log entry looks like. Between them, processInternalRaftRequestOnce is the gatekeeper that admits or rejects each proposal, assigns it an ID, and parks the calling goroutine on a wait channel, while raftNode.start is the long-running loop that talks to the Raft library and turns its output into work for the rest of the server. Read.LinearizableReadLoop is the read-side mirror of all this: it exists so that reads can be linearizable without going through the log themselves, and the graph records exactly one caller for it, EtcdServer.Start, which launches it as a goroutine once at boot.

How it works

Here is the canonical path of a client Put, from the gRPC handler to the applied mutation and back.

1. EtcdServer.Put receives the decoded PutRequest from the gRPC layer (v3rpc.kvServer.Put validates it first), wraps it in an InternalRaftRequest (the union type every mutating operation shares), and calls raftRequest, which delegates to processInternalRaftRequestOnce on its first line and afterwards reworks the traceutil.Trace carried back on the result so a slow write gets logged.
2. EtcdServer.processInternalRaftRequestOnce runs the admission check exceedsRequestLimit first, which compares the server's applied and committed Raft indexes so a client cannot propose faster than the node can apply, and only then stamps the request with a unique ID from idutil.Generator.Next, resolves the caller through AuthInfoFromCtx, and classifies it with getRequestType. It marshals the request, registers a wait keyed on the request ID, and proposes the bytes to the Raft node.
3. raftNode.start is where the proposal leaves etcd's own code. The Raft library replicates the entry to peers over the transport, and once a quorum has acknowledged it, the library returns it as a committed entry on its Ready channel. This loop reads each Ready, packages its committed entries as a toApply batch and sends it down a channel to the server, sends outbound messages to peers, and persists the unstable entries and hard state to the write-ahead log through the server's Storage wrapper; on the leader path the toApply handoff and the peer send both go out before the disk write, so persisting overlaps with replication to the followers. Because this hop is a channel handoff rather than a call, the static graph shows no edge from the propose side to the apply side.
4. EtcdServer.apply drains that batch inside the server's own run loop (the graph reaches it through EtcdServer.run and applyEntries). It walks the committed entries in index order, which is the point where Raft's total ordering becomes etcd's execution order.
5. EtcdServer.applyEntryNormal handles each ordinary entry: it unmarshals the bytes back into an InternalRaftRequest, checks whether the entry has already been applied (so replay after a crash is idempotent), and dispatches it through the applierV3 interface rather than a direct call.
6. applierV3backend.Put is the applier method the dispatch lands on for a Put. This is the first moment in the entire path that the request is actually a write against storage rather than a proposal about a write.
7. storeTxnWrite.Put does the mutation. The applier delegates to txn.Put, which checks the lease and opens a write transaction with store.Write, and storeTxnWrite.Put assigns the key its new main revision, writes the versioned value into the bolt-backed store, and updates the in-memory treeIndex. When the transaction ends and the apply index advances, the wait registered back in step 2 fires and the original Put call unblocks with its response.
8. Read.LinearizableReadLoop is how reads stay consistent with all of the above without paying for a log entry. A linearizable read requests a confirmed read index from the leader, records the current AppliedIndex, and if the applied index is behind, waits on ApplyWait until the store has caught up to that index before serving from mvcc. It batches concurrent reads so one round trip to the leader can release many waiters at once.

Where it connects

This path is the spine the hub page's modules hang off. Steps 1 through 5 all live inside etcdserver, the largest module on the etcd module map at 962 symbols, and the hub of that module is raftRequest, the call step 1 makes. The request and option vocabulary a client uses to build the Put in the first place lives in client/v3, the largest module outside the server; the durability step in the middle belongs to wal; and the applied mutation in step 7 lands in mvcc. The apply loop is the only writer into that storage, which is why neither applierV3backend.Put nor storeTxnWrite.Put carries a call edge back to the code that drives it: each is reached through an interface (applierV3 for the applier, mvcc.TxnWrite for the store transaction), and an interface dispatch is a runtime decision the static graph cannot resolve to a single target.

The half of the story this spoke skips over is step 3's phrase "persists the entries to the write-ahead log." Before Raft is allowed to call an entry committed and before the apply loop is allowed to execute it, that entry has to be durable on disk, so a power loss cannot lose an acknowledged write. That durability contract, the encoding, the fsync discipline, the segment rotation, and the replay that rebuilds this exact in-memory state after a restart, is the subject of the sibling spoke on the write-ahead log.

By the numbers

The propose side of the flow is compact: processInternalRaftRequestOnce spans 76 lines and, per the graph, pulls in 54 symbols, among them exceedsRequestLimit, getAppliedIndex, getCommittedIndex, AuthInfoFromCtx, getRequestType, parseProposeCtxErr, and the three sentinel errors it can return without ever reaching Raft (ErrTooManyRequests, ErrRequestTooLarge, ErrStopped). The apply side is where the length lives: applyEntryNormal runs 59 lines and raftNode.start, the ready loop, is a single 169-line function (raft.go lines 174 to 342) that pulls in 59 symbols of its own: the external go.etcd.io/raft/v3 v3.7.0 API (IsEmptySnap, CommittedEntries, SoftState, StateLeader), etcd's own storage wrapper (Save, SaveSnap, Release, Sync), and the peer transport's Send. The clean split between a thin proposal front end and a fat apply back end is the structural signature of a consensus system: almost nothing happens where the request arrives, and everything happens where the log is replayed.