Reliability · advanced
Retries, exponential backoff, and jitter
A retry policy repeats eligible failures within an attempt or deadline budget. Capped exponential backoff grows the maximum delay window between attempts; jitter randomizes the actual wait within that window so many clients do not retry in lockstep.
Why it matters
Retries can hide transient faults, but every attempt adds load and latency. A careless policy can amplify an overload, repeat a successful side effect after an ambiguous timeout, and consume the caller's entire deadline before returning a useful error.
Mental model
How to reason about retries, exponential backoff, and jitter
Treat a retry as a new request that spends remaining time and adds load. Before each attempt require four gates: the failure is likely transient, repetition is safe or deduplicated, enough end-to-end deadline remains, and the system permits more retry traffic. Then use capped exponential spacing with jitter.
Analogy
If a busy office line fails, immediate repeated calls join everyone else's redial storm. A sensible caller retries only calls worth repeating, waits within progressively wider randomized windows, and stops before the conversation's time budget expires.
Examples
See the boundary, not just the happy path
Worked example · Calculate full-jitter delay
ceiling = min(5s, 100ms × 2^n)
sleep = random(0, ceiling)Let n = 0 for the first retry. The delay ceiling grows exponentially until the 5-second cap, while the sampled wait can be anywhere from zero to that ceiling and can be shorter than the previous sample.
Worked example · Classify failures before retrying
consider retry: connection reset, selected 429/503
normally stop: unchanged 400/401/403Eligibility is part of the API contract and operation context, not just a status-code list. Retry an ambiguous timeout only when the operation is idempotent, deduplicated, or safely reconciled.
Worked example · Respect server guidance
HTTP/1.1 429 Too Many Requests
Retry-After: 120Retry-After can be delay-seconds or an HTTP date. Treat it as server guidance, but do not wait and retry if the caller's remaining deadline or retry budget cannot accommodate another attempt.
Worked example · Protect a repeated write
POST /payments
Idempotency-Key: 8f4b7c1e-...A server-defined idempotency-key contract can recognize a repeated logical operation after a lost response and return the recorded result instead of creating a second payment.
Worked example · See retry amplification
5 call layers × 3 total attempts per layer → up to 3^5 = 243 deepest-layer callsIndependent retries at every layer multiply downstream work. A deliberate retry boundary near the caller avoids turning one failing request into a retry storm.
Avoid · Retry a permanent client error
repeat the same malformed request after HTTP 400Nothing changes between attempts, so retries waste capacity and delay the actionable error.
Common mistakes
Misconceptions to remove early
Retrying at every service layer
Nested policies multiply attempts rather than add them. Choose a deliberate retry boundary and account for retries already performed by SDKs, proxies, queues, and downstream clients.
Retrying non-idempotent work blindly
A timeout does not prove the server did nothing. Use idempotency, deduplication, or an operation-specific reconciliation strategy before repeating a write.
Capping backoff without adding jitter
Clients that use the same deterministic schedule can synchronize on every retry and especially at the cap. Randomize the actual wait to spread competing attempts over time.
Counting attempts without an end-to-end deadline
A small attempt count can still exceed the user's patience when per-attempt timeouts and backoff accumulate. Check remaining deadline before starting each attempt and reserve time to return or recover.
Quick check
Can you predict the result?
1. What problem does jitter primarily address?
- • Many clients retrying at the same scheduled moments and causing synchronized load spikes
- • Incorrect JSON serialization
- • A mutex being unlocked twice
2. With base 100ms, n = 2, and a 1s cap, what range can full jitter sample?
- • 0 through 400ms
- • Exactly 400ms
- • 400ms through 1s
3. Why does a timeout not prove that retrying a write is safe?
4. What four questions should a retry policy answer?
Keep building