Concept · Consistency & coordination

Gossip protocols

Spread membership and state by having each node talk to a few random peers.

2 mingossipepidemic protocolSWIMfailure detection

In a gossip (epidemic) protocol, every node periodically exchanges information with a few randomly chosen peers. Each exchange merges what both know, so news spreads through the cluster like a rumour, reaching all N nodes in roughly O(log N) rounds — without any central coordinator.

What it's used for

  • Membership: which nodes exist and which are alive. Cassandra, ScyllaDB, Consul (through Serf/memberlist) and many service meshes use gossip for this.
  • Failure detection: protocols such as SWIM probe a random peer each round; if it doesn't answer, they ask a few other nodes to probe it indirectly before suspecting it, which avoids false alarms caused by one bad link.
  • Spreading small state: node load, schema versions, token ownership on a hash ring (consistent hashing).

Why it works well

  • Scales: each node does a constant amount of work per round regardless of cluster size.
  • Robust: no single point of failure; messages lost in one round are covered by later ones.
  • Simple: merging state with version numbers ("keep the higher version") makes exchanges idempotent and order-independent.

Limits

  • Eventually consistent: at any moment, different nodes may hold different views. Fine for "who's alive", not for "who's the leader" or "has this transaction committed" — those need consensus (consensus and Raft).
  • Propagation delay: seconds rather than milliseconds in large clusters.
  • Bandwidth grows with the amount of state gossiped, so keep the gossiped payload small.

Anti-entropy repair in leaderless databases is a close cousin: replicas periodically compare data and exchange what's missing, often using Merkle trees.

Learn it in depth