Concept · Requirements & scale
Latency numbers to know
Memory, SSD, network and cross-region round trips, to the nearest order of magnitude.
2 minlatency numbers every programmer should know
You don't need exact figures to design well, but you do need to know which operations differ by a factor of a thousand. These are order-of-magnitude values for modern hardware.
| Operation | Approximate time |
|---|---|
| CPU L1 cache reference | ~1 ns |
| Main memory reference | ~100 ns |
| Compress 1 KB (fast codec) | ~1–2 µs |
| Read 1 MB sequentially from memory | ~10–50 µs |
| NVMe SSD random read (4 KB) | ~20–100 µs |
| Round trip within a data centre | ~0.1–0.5 ms |
| Read 1 MB sequentially from NVMe SSD | ~0.2–1 ms |
| Redis GET over the network, same zone | ~0.2–1 ms |
| Simple indexed database query, same zone | ~1–5 ms |
| Spinning disk seek | ~5–10 ms |
| Round trip, same continent | ~20–80 ms |
| Round trip, across an ocean | ~80–150 ms |
| TLS handshake to a distant server | ~2 round trips on top of TCP's 1 (TLS 1.2), 1 with TLS 1.3 |
What the table tells you
- Memory is about 1,000× faster than a network hop. A cache is only worth it if the work it saves is slower than the network call to the cache.
- Round trips dominate distributed latency. A request that makes ten sequential calls inside a data centre spends 1–5 ms just waiting on the network. Parallelise or batch.
- Distance is physics. Light in fibre covers roughly 200 km per millisecond, so a London–Sydney round trip can never be under about 170 ms. Put data and caches near users (content delivery networks).
- Sequential beats random. Logs, append-only storage and batch reads exist because sequential I/O is far cheaper than random I/O (B-trees vs LSM-trees).
Use these numbers in capacity estimation and when a design adds a hop, ask what it costs.