Retry with Backoff
What it is
A failed request often works if you try it again, a little later. Retrying immediately can make a busy service even busier. Waiting longer after each failure reduces that pressure. Adding a random amount to each wait, instead of always waiting the same calculated length, stops many clients from retrying at the exact same moment. This trick has a name: jitter.
The core idea
Exponential backoff makes the longest allowed wait grow quickly after repeated failures, roughly doubling each time. A cap keeps that longest wait from growing forever. Full jitter then picks the actual wait as a random amount anywhere between zero and that longest allowed wait, rather than always waiting the full amount. That is one good style, not the only one. Equal jitter waits half the longest allowed wait, plus a random amount up to the other half. It also spreads the waits out. Longer waits reduce pressure on a struggling service, but also delay recovery for the caller trying it. A retry limit bounds how much work, and how much waiting, one failing request can cause.
Example
With a starting wait of 8 ms and a cap of 40 ms, attempt 0 can wait anywhere from 0 to 8 ms. Attempt 3 can wait anywhere from 0 to 40 ms, because doubling three times (8, 16, 32, 64 ms) would go past the 40 ms cap, so it is held at the cap instead. Later attempts stay at that same 40 ms cap. Across many calls at the same attempt, the waits should spread out across that whole range, not cluster near one number.
Why randomness matters here
Picture ten clients that all call the same struggling service and all fail at the same moment. Without randomness, all ten compute the exact same wait and all retry at the exact same moment too, hitting the service with another wave just as hard as the first. This is sometimes called a thundering herd. Adding a random wait scatters those ten retries out over time instead, so the service sees a trickle of retries rather than another wave.
Where you meet this
- AWS SDKs, in the standard retry mode described in their 2026 retry guide (opt-in for now, with
AWS_NEW_RETRIES_2026=true), usedelay = random(0, 1) * min(20 s, base * 2^retry), which is full jitter. They also keep a retry budget: when most calls are failing, they stop retrying and return the error fast. The older legacy mode differs between SDKs, and a wait the service asks for (x-amz-retry-after) gets no jitter. (AWS SDK retry behavior) - The AWS Architecture Blog post that compares the jitter styles shows why backoff alone is not enough: clients that fail together still retry in clusters. (Exponential Backoff and Jitter)
- gRPC reconnects with a 1 second first wait, a growth factor of 1.6, 20% jitter, and a 120 second limit. (gRPC connection backoff)
When to choose it: the error is likely to pass, like a timeout, a dropped connection, or "too busy, slow down".
When not to: the error will not pass on its own, like bad input or "access denied". The AWS SDK standard mode returns those at once without a retry. Also be careful with a request that changes something. A retry can do it twice unless the server supports an idempotency key.
Common bugs
- Giving up after the first failure, instead of retrying.
- Making one extra attempt, or stopping one attempt early.
- Forgetting the cap, so the wait keeps doubling forever.
- Always waiting the full allowed amount instead of a random amount within it.
- Calculating one random wait and reusing it for every call.
- Sharing one random number generator across callers without protecting it from concurrent use.
Keep it fast
Work out the wait with a small amount of math, not a long loop. Avoid creating new memory on every single wait you compute.
Learn more
The AWS Architecture Blog's "Exponential Backoff and Jitter" post explains full jitter in detail. A circuit breaker decides whether to call a service at all. A retrier decides when to try again.