Hot partitions and hot keys

Partitioning spreads load only if load is spread across keys. When one key — a celebrity's profile, a viral post, a flash-sale product, one enormous tenant — gets a large share of the traffic, its partition (or cache node) saturates while others idle. Adding machines doesn't help, because every request for that key goes to the same place.

#Detecting them

• Track request rate and latency per partition, and alert when one is far above the median.
• Sample keys and count the most frequent ones (a count-min sketch finds heavy hitters in fixed memory).
• Watch for throttling errors concentrated on one partition (DynamoDB reports these explicitly).

#Fixes for hot reads

• Cache the hot key, including a short-lived in-process cache on each server — even one second of local caching turns 100,000 reads per second into one per server per second.
• Replicate the key under several names (item:42#0 … item:42#7) and read one at random.
• Serve from a CDN when the response is public.

#Fixes for hot writes

• Salt the key: write to counter:post:42#n with a random n from 0–9, spreading writes over ten partitions; reads sum all ten. Works well for counters and append-only data, poorly for read-modify-write.
• Buffer and batch: aggregate increments in memory or a write-behind cache and flush periodically (caching patterns).
• Split the entity: partition a huge conversation or tenant by a second key, such as (conversation_id, month).
• Isolate big tenants on dedicated partitions or clusters.

#Prevent them with key design

• Avoid keys whose values are naturally skewed (country, status, "today's date").
• Avoid monotonically increasing keys in range-partitioned stores — every write hits the newest range (range vs hash partitioning).

The news-feed lesson's "celebrity problem" is the same issue at the application level, solved by pulling instead of pushing for the hottest accounts. See also cache stampedes.