Timeouts and deadlines

A call without a timeout can wait forever. When a dependency hangs, every caller's thread, connection and memory for that request stays tied up, and by Little's law the number of stuck requests grows until the caller runs out of resources too. Timeouts are the first line of defence against cascading failure.

#Choosing values

• Base them on measurements: a little above the dependency's p99.9 latency under normal load. If it answers in 40 ms at p99.9, 200 ms is generous; 30 seconds is a liability.
• Set connect and request timeouts separately. Connecting should take milliseconds inside a data centre; a long connect wait usually means the target is gone.
• Make an optional call's timeout short and give it a fallback (graceful degradation).

#Deadlines propagate

A user request has an overall budget. Each downstream call should get what's left, not a fresh full timeout; otherwise a chain of services can keep working long after the user has given up.

// The handler's budget: 1 s. Every call made with ctx inherits the remaining time.
ctx, cancel := context.WithTimeout(r.Context(), time.Second)
defer cancel()

user, err := users.Get(ctx, id)                 // may use up to the full second
recent, err := orders.ListRecent(ctx, user.ID)  // gets whatever remains

gRPC transmits deadlines to the server automatically, so a downstream service can stop work its caller has abandoned. With plain HTTP, pass the remaining budget in a header and apply it on the server.

#Don't forget

• Database queries (statement_timeout in PostgreSQL), lock waits (lock_timeout) and connection-pool acquisition need limits too.
• In Go, http.Client{} has no timeout by default; always set one or use a context.
• A timeout means the outcome is unknown, not failed: the server may have completed the work. Retrying a non-idempotent call after a timeout needs an idempotency key.

See retries, backoff and jitter and circuit breakers.