Replication lag anomalies

Asynchronous replicas trail the leader. Usually the lag is milliseconds; under heavy writes, long queries on a replica or network trouble it can be seconds or minutes. Reading from a lagging replica produces three well-known anomalies.

#Read-your-writes violation

A user saves a change, the next page reads from a replica that hasn't applied it, and the change seems lost.

Fixes:

• Read from the leader for a short window (say 5–10 seconds) after a user writes.
• Track the leader's log position after the write and only read from replicas that have reached it. In PostgreSQL:

-- On the leader, after the write commits: remember this value for the user.
SELECT pg_current_wal_lsn();
-- On a candidate replica before reading:
SELECT pg_last_wal_replay_lsn() >= '3/A1C2B8F0'::pg_lsn;

#Non-monotonic reads

Two consecutive reads hit different replicas; the second is further behind, so data appears to go backwards in time.

Fix: pin each user (or session) to one replica, for example by hashing the user ID, so their reads only move forward.

#Inconsistent prefix (broken causality)

A reader sees an effect before its cause — a reply before the question — because related writes reached the replica out of order. This mostly happens in partitioned systems, where different partitions replicate independently.

Fix: keep causally related data in the same partition, or read it from the leader, or use a store with causal consistency (consistency models).

#Operating with lag

• Monitor lag in time and bytes (now() - pg_last_xact_replay_timestamp() on a PostgreSQL replica) and alert on it.
• Stop routing reads to a replica whose lag exceeds a threshold.
• Run heavy analytics on a dedicated replica so it can't slow the ones serving users.

See leader–follower replication and the System Design roadmap's replication lesson.