# Tail latency

> Why p99 matters more than the average, how fan-out amplifies it, and hedged requests.

Concept · Requirements & scale · https://levelcode.org/concepts/tail-latency

Tail latency is the latency of the slowest requests: the 99th percentile (p99) or 99.9th (p999). Averages hide it, and at scale it's what users actually notice.

## Why the tail matters

- **Heavy users see it most.** Someone who loads 100 pages a day hits the p99 about once a day, and those users are often your best customers.
- **Fan-out amplifies it.** If a page needs answers from 50 backend calls in parallel and each has a 1% chance of being slow, the chance that at least one is slow is:

```text
1 − 0.99^50 ≈ 0.39  → 39% of page loads see a p99-slow call
```

The more a request fans out, the more the backend's tail becomes the user's median.

## Where tail latency comes from

Garbage-collection pauses, cache misses, a noisy neighbour on a shared host, queueing behind a large request, a compaction or vacuum running, a retransmitted TCP packet, a cold connection needing a TLS handshake. Most are brief and random, which is why the techniques below work.

## Reducing it

- **Hedged requests.** Send the request to one replica; if no answer arrives within the p95 latency, send it to a second and use whichever answers first. It adds a few percent of load and can cut p99 dramatically. Only do this for idempotent reads.
- **Tighter timeouts with fallbacks** for optional calls, so one slow dependency doesn't hold the page ([timeouts and deadlines](https://levelcode.org/concepts/timeouts)).
- **Reduce fan-out** by batching calls or precomputing results.
- **Keep queues short** and separate large requests from small ones, so small ones don't wait behind big ones.
- **Measure correctly.** Record latency in histograms and compute percentiles over all requests. Averaging p99 values from several hosts doesn't give the fleet's p99.