# Design a chat system

> Design one-to-one and group messaging: persistent connections, sending and fan-out, per-conversation ordering, storage, multi-device sync, receipts, presence and push notifications.

System Design roadmap · Case studies · 21 min read · https://levelcode.org/roadmaps/system-design/design-chat-system

Chat — WhatsApp, Slack, Discord, Messenger — feels simple to use and is demanding to build. Messages must arrive in well under a second, never be lost, appear in the same order for everyone in a conversation, and sync across a user's phone and laptop, while tens of millions of devices hold open connections at once. This lesson designs a chat backend, focusing on the parts that are specific to messaging: long-lived connections, ordering, delivery and sync.

## Requirements

**Functional:**

- One-to-one and group conversations (up to 500 members).
- Text messages and attachments (images, files).
- Delivery state per message: sent, delivered, read.
- Each user can have several devices; every device shows the full history.
- Presence ("online", "last seen") and push notifications for offline devices.
- Out of scope: voice and video calls, end-to-end encryption (which changes the server's role to relaying ciphertext, but not the delivery design).

**Non-functional:**

- p99 delivery under 500 ms between online users in the same region.
- **No message loss** once the server has acknowledged it.
- **Consistent order** within a conversation, the same on every device.
- High availability: users should be able to send even during partial failures.

## Numbers

```text
daily active users:        100 million
messages:                  40 per user per day → 4 billion/day ≈ 46,000/s avg, ~150,000/s peak
concurrent connections:    ~30% of DAU online at peak → 30 million WebSockets
message size:              ~200 bytes of text and metadata → 800 GB/day, ~300 TB/year
```

Two numbers shape the design. 30 million concurrent connections means connection handling is its own tier. And 300 TB a year of append-mostly, time-ordered data read per conversation points to a wide-column store rather than a single relational database.

## Connections: keeping a line open

Chat needs the server to push messages to clients the moment they arrive. Polling every few seconds is too slow and wasteful at this scale, so clients hold a **WebSocket** open (with server-sent events or long polling as fallbacks). See [polling, SSE and WebSockets](https://levelcode.org/concepts/realtime-transports).

That makes the edge of the system **stateful**: a dedicated tier of **gateway servers** terminates WebSockets. Each holds tens or hundreds of thousands of connections (an idle connection costs memory for buffers and TLS state, roughly 10–50 KB), so 30 million connections need a few hundred gateways. Gateways do as little as possible: authenticate the connection, pass messages to the chat service, and deliver messages back to devices.

To deliver a message to a device, the system must know which gateway holds its connection. A **connection registry** maps `device_id → gateway_id`, stored in Redis with a short TTL that each gateway refreshes with heartbeats. If a gateway dies, its entries expire and its clients reconnect elsewhere.

> [!WARNING]
> When a gateway restarts, all its clients reconnect at the same moment. Without randomised backoff on the client, they hammer the remaining gateways and authentication service together. Clients should reconnect with exponential backoff and full jitter ([retries, backoff and jitter](https://levelcode.org/concepts/retries-with-backoff)), and gateways should be drained gradually during deploys.

## Sending a message

Every message carries a **client-generated ID** so retries are safe: if the sender's connection drops before it hears the acknowledgement, it resends with the same ID and the server deduplicates ([idempotency keys](https://levelcode.org/concepts/idempotency-keys)).

```sequence
title: Alice sends a message to Bob
participants: Alice's phone, Gateway A, Chat service, Message store, Gateway B, Bob's laptop
Alice's phone -> Gateway A: send {conv 77, client_id m-91, "hi"}
Gateway A -> Chat service: forward
Chat service -> Message store: append with seq = 1042 (dedupe on client_id)
Message store --> Chat service: stored
Chat service --> Gateway A: ack {client_id m-91, seq 1042}
Gateway A --> Alice's phone: sent ✓
Chat service -> Gateway B: deliver to Bob's devices (via registry)
Gateway B -> Bob's laptop: message seq 1042
Bob's laptop --> Gateway B: delivered ack
```

The order matters: the server acknowledges to the sender **only after the message is durably stored**. That's what "no loss once acknowledged" means. Delivery to recipients happens after, asynchronously; a recipient that's offline will get the message on its next sync.

For devices with no open connection, the chat service hands the message to a **push notification service**, which sends through Apple Push Notification service or Firebase Cloud Messaging. Push is best-effort and rate-limited by the platforms, so it's a wake-up signal ("you have a new message"), never the delivery mechanism of record.

## Ordering within a conversation

Clocks on phones are unreliable, and two servers' clocks differ (see [clocks and ordering](https://levelcode.org/concepts/clocks-and-ordering)), so timestamps can't order messages. Instead, give each conversation a **sequence number** assigned by one place:

- Partition conversations by `conversation_id` across chat-service instances, so each conversation has exactly one owner at a time (via consistent hashing or a shard map, with a lease to prevent two owners — see [leases and fencing tokens](https://levelcode.org/concepts/leases-and-fencing-tokens)).
- The owner assigns `seq = last_seq + 1` as it stores each message. Every device sorts by `seq`, so everyone sees the same order.
- A device that receives `seq 1045` while its latest is `1042` knows it missed two messages and fetches them.

Ordering is per conversation only. Messages in different conversations have no meaningful relative order, which is what lets the system scale: conversations are independent.

## Storing messages

The dominant query is "the latest N messages in conversation C, and those before sequence S" — a range scan within one conversation. That's an ideal fit for a wide-column store such as Cassandra or ScyllaDB (Discord moved its message storage from Cassandra to ScyllaDB at trillions of messages):

```sql
-- CQL. One partition per conversation per bucket keeps partitions bounded in size.
CREATE TABLE messages (
  conversation_id bigint,
  bucket          int,          -- e.g. a 10-day window, so huge conversations don't make one giant partition
  seq             bigint,
  message_id      text,         -- the client-generated ID, for deduplication
  sender_id       bigint,
  body            text,
  attachment_key  text,
  created_at      timestamp,
  PRIMARY KEY ((conversation_id, bucket), seq)
) WITH CLUSTERING ORDER BY (seq DESC);
```

Reading the latest page is one partition read, sorted. Attachments go to [object storage](https://levelcode.org/concepts/object-storage) — the client uploads with a presigned URL and sends a message containing only the object key — so the message path stays small.

Per-user state lives separately: each user's conversation list (with the latest message and unread count for the sidebar), and per-user, per-conversation **read pointers** (`last_read_seq`).

## Multi-device sync and receipts

Every device keeps, per conversation, the highest `seq` it has. Sync is then simple and robust:

- On connect, the device sends its cursors; the server returns everything newer, a page at a time. A device that was offline for a week catches up the same way as one that missed a single message.
- Live messages arrive over the WebSocket; if one is missed, the gap in `seq` triggers a fetch.
- Delivery is effectively at least once (a message can arrive both live and via sync), and devices deduplicate by `seq`.

**Receipts** are just more state:

- **Sent:** the sender received the server's acknowledgement.
- **Delivered:** a recipient's device acknowledged the message. Report it to the sender as a small event.
- **Read:** the recipient's read pointer moved past the message. Store `last_read_seq` per user per conversation, and send it to the other members' devices — in large groups, as an aggregate ("read by 12") rather than one event per reader.

## Group fan-out and presence

For a group, the chat service stores each message once and then delivers it to every member's online devices through their gateways. With up to 500 members, fan-out on write is fine: up to a few thousand deliveries per message. Very large rooms (Discord servers or Slack channels with hundreds of thousands of members) switch to the pull model of the news-feed lesson: clients viewing the channel subscribe to it, and others fetch on open.

**Presence** is high-volume and low-value per event, so keep it cheap and [eventually consistent](https://levelcode.org/concepts/consistency-models):

- Devices send heartbeats every 30 seconds; a device is online while its heartbeat key in Redis hasn't expired.
- Don't broadcast every change to every contact. Clients subscribe to presence only for contacts currently on screen, and changes are batched.
- "Last seen" is written at most every few minutes.

## When things fail

| Failure | Effect | Handling |
|---|---|---|
| A gateway dies | Its clients disconnect | Clients reconnect to another gateway with jittered backoff and sync from their cursors |
| Chat-service owner of a conversation dies | Sends to it fail briefly | Another instance takes the lease; clients retry with the same client ID |
| Message store slow | Sends slow down | Senders see "sending…"; the client queues locally and retries; no ack until stored |
| Push provider down | Offline users aren't woken | Messages are still stored; they're delivered on next open |

## Practice

Design two features on top of this system:

1. **Typing indicators** ("Bob is typing…") in one-to-one and group chats.
2. **Delete for everyone**: a sender can delete a message up to an hour after sending, and it must disappear from every member's devices, including devices that are offline right now.

For each, say what's stored, what's sent over the connection, and what consistency is acceptable.

**Solution**

**1. Typing indicators.** These are ephemeral and losing one is harmless, so **don't store them**. While the user types, the client sends a `typing` event to its gateway at most every 3 seconds; the chat service forwards it to the conversation members' *online* devices only, with no persistence, no acknowledgements, no retries and no push notifications. Receiving clients show the indicator and hide it after 5 seconds without a fresh event (or when a message from that user arrives). In large groups, cap it ("several people are typing") and only forward to members who currently have the conversation open, to avoid fan-out cost. At-most-once delivery is the right guarantee.

**2. Delete for everyone.** This must reach every device, including offline ones, so it goes through the **same ordered, durable path as a message**. The client sends `delete {conversation, target_seq}`; the chat service checks that the requester is the sender and that the message is less than an hour old, then appends a **tombstone event** with its own new sequence number (say seq 1101 deleting seq 1042) and marks the original as deleted in the store (removing its body and attachment reference, and scheduling the object for deletion). Online devices receive the tombstone live and remove the message; offline devices receive it during sync, because it's just another entry after their cursor. Devices that later fetch history get the original as "deleted". This is consistent with the conversation's order: every device applies the deletion at the same point in the sequence.

## Key takeaways

- Push delivery needs persistent connections; run a dedicated gateway tier and a connection registry that maps devices to gateways.
- Acknowledge a send only after the message is durably stored; deduplicate retries with client-generated IDs.
- Order messages with a per-conversation sequence number from a single owner, not timestamps; gaps tell clients what they missed.
- Store messages partitioned by conversation (and time bucket), clustered by sequence, in a wide-column store; attachments go to object storage.
- Multi-device sync is cursors per conversation; receipts are read pointers and acknowledgements.
- Treat push notifications as wake-ups, keep presence cheap and eventually consistent, and have clients reconnect with jitter.

## Quiz

1. When should the chat service acknowledge a message to its sender?

   A. As soon as the gateway receives it
   B. After every recipient has received it
   C. After the message is durably stored
   D. After the push notification is sent

   **Answer: C.** Acknowledging before storage risks losing an acknowledged message; waiting for every recipient makes sending depend on offline devices. Durable storage is the point after which delivery can always be completed.

2. Why does each conversation get a server-assigned sequence number instead of ordering by timestamp?

   A. Timestamps take more storage
   B. Device and server clocks differ and drift, so timestamps can order messages differently on different devices; a single sequencer gives everyone the same order
   C. Sequence numbers are required by WebSockets
   D. Timestamps can't be indexed in Cassandra

   **Answer: B.** One owner per conversation assigning seq = last + 1 defines a single order. Gaps in the sequence also let a device detect and fetch messages it missed.

3. A laptop has been offline for three days. How does it catch up?

   A. The server pushes three days of messages through APNs
   B. It sends its last seen sequence number for each conversation and fetches everything newer, page by page
   C. It re-downloads every conversation from the beginning
   D. Messages older than one day are lost

   **Answer: B.** Per-conversation cursors make sync uniform: a device that missed one message and one that missed thousands use the same request. Push notifications are only wake-ups.

4. Why must clients reconnect with randomised backoff after a gateway restarts?

   A. WebSockets can only reconnect once per minute
   B. Otherwise all its clients reconnect at the same instant and overload the remaining gateways and the authentication service
   C. Jitter makes messages arrive in order
   D. It reduces the size of each message

   **Answer: B.** A gateway with 100,000 connections becomes 100,000 simultaneous reconnects. Spreading them over time with jittered backoff turns a spike into a manageable ramp.