Concept · Consistency & coordination
Leases and fencing tokens
A lock that expires, plus a number that lets storage reject the holder whose lease already ran out.
A lease is a lock with an expiry time. If the holder crashes, the lease expires on its own and someone else can take it — unlike a plain lock, which a dead holder keeps forever. Leases are the basis of leader election and distributed locks.
The problem leases can't solve alone
1. Client A acquires the lease (valid 10 s)
2. A pauses for 15 s (GC, VM migration, swap)
3. The lease expires; client B acquires it and writes to storage
4. A resumes, believes it still holds the lease, and writes to storageNow both A and B have written as "the holder". No lease length prevents this, because a pause can always be longer, and checking the lease right before writing doesn't help — the pause can happen between the check and the write.
Fencing tokens
Each time the lease is granted, the lock service hands out a token that only increases: 33, then 34, and so on. The holder sends its token with every write, and the storage remembers the highest token it has seen and rejects anything lower.
A holds token 33 · B later gets token 34
B writes with 34 → accepted; storage remembers 34
A wakes, writes with 33 → rejected (33 < 34)The protection lives where the damage would happen: in the resource itself.
Implementations
- etcd: use the key's revision (or the lease's creation revision) as the token.
- ZooKeeper: the sequence number of the lock's ephemeral node, or the
zxid. - A SQL table as the resource: a conditional update,
UPDATE jobs SET …, fence = $token WHERE id = $id AND fence < $token, and check that one row changed.
About Redis locks
A lock built only from SET key value NX PX ttl (or the multi-node Redlock algorithm) has an expiry but no fencing token, and its safety depends on timing assumptions. It's fine for efficiency — avoiding duplicate work most of the time — where an occasional double execution is harmless. For correctness, use a consensus-backed lock with fencing, and make the protected operation idempotent as well. See consensus and Raft.