# Cache stampedes

> One expiring hot key, a thousand simultaneous misses — and three ways to stop it.

Concept · Caching · https://levelcode.org/concepts/cache-stampede

A cache stampede (thundering herd, dogpile) happens when a popular key expires or is evicted and many concurrent requests miss at the same moment. All of them recompute the value or query the database for the same row, and the burst can overload the backend the cache was protecting.

## Request coalescing

Let only one request per key load the value; the others wait for its result.

```go
var group singleflight.Group // golang.org/x/sync/singleflight

func (c *Cache) Get(ctx context.Context, key string) (Product, error) {
	if p, ok := c.lookup(ctx, key); ok {
		return p, nil
	}
	v, err, _ := group.Do(key, func() (any, error) {
		p, err := c.db.Load(ctx, key)
		if err == nil {
			c.store(ctx, key, p)
		}
		return p, err
	})
	if err != nil {
		return Product{}, err
	}
	return v.(Product), nil
}
```

`singleflight` works within one process; across N servers you get at most N loads. A short-lived distributed lock (`SET lock:key 1 NX PX 5000` in Redis) extends it across servers, with waiters retrying the cache.

## Refresh before expiry

- **Refresh-ahead:** a background job reloads known hot keys before their TTL ends.
- **Probabilistic early expiration:** each read recomputes early with a probability that rises as expiry approaches, so typically one request refreshes in time and nobody sees a miss.
- **Stale-while-revalidate:** keep serving the expired value while one request refreshes it in the background. HTTP supports it directly: `Cache-Control: max-age=60, stale-while-revalidate=30`.

## Spread out expiry

Keys written together with the same TTL expire together. Add random jitter (±10%) to TTLs so a cache warm-up doesn't become a synchronised wave of misses later.

## Also plan for the empty cache

A cache restart is a stampede on every key at once. Make sure the backend survives a cold cache (or warm it first), and use [load shedding](https://levelcode.org/concepts/load-shedding) to cap what reaches the database while the cache refills. See [hot partitions](https://levelcode.org/concepts/hot-partitions) for a single key that's hot all the time.