Exponential Backoff: Retrying Failed Requests the Right Way

Quick Answer

Exponential backoff means waiting longer before each retry: roughly base * 2^attempt, so 1s, 2s, 4s, 8s and so on, up to a cap. It gives a struggling service room to recover instead of being hammered. On its own it is not enough - you also add jitter (randomness) so that many clients which failed at the same moment do not all retry at the same moment, and you only retry errors that could plausibly succeed on a repeat, like timeouts and 503s, never a 400.

Why a tight retry loop makes things worse

A service returns 503 because it is overloaded. Your code retries immediately. So do the other 5,000 clients that also just got a 503. The service now receives its normal load plus 5,000 instant retries, becomes more overloaded, returns more 503s, and triggers even more retries. This is a retry storm, and it is a common way a brief blip turns into a sustained outage.

Backoff breaks the loop by waiting - and waiting longer each time. The first retry after one second, the next after two, the next after four. Total pressure on the service drops sharply, and the growing gaps give it time to work through its backlog.

The instinct to "just try again right away" is exactly backwards: the moment right after a failure is the worst possible time to retry, because everything else is retrying then too.

The formula, and why you cap it

The delay before attempt n is min(cap, base * 2 ** n), with base a starting delay (100ms to 1s is typical) and n starting at 0.

function baseDelay(attempt, { base = 200, cap = 20_000 } = {}) {
  return Math.min(cap, base * 2 ** attempt);
}

attempt : uncapped   : capped at 20s
   0    :     200ms  : 200ms
   3    :    1600ms  : 1600ms
   6    :   12800ms  : 12800ms
   7    :   25600ms  : 20000ms
   9    :  102400ms  : 20000ms

The cap matters because 2 ** n grows fast. By attempt 9 an uncapped 200ms base is already over 100 seconds; a few more and it is close to an hour. Without a cap a client ends up "retrying" a request it first made an hour ago, long after the user gave up and left. A cap of 20 to 60 seconds keeps the longest wait sane while still being gentle. Cap the number of retries too (three to five is normal), and give the whole operation an overall deadline so it cannot hang forever.

Jitter: the part everyone forgets

Backoff alone does not fix the storm. If 5,000 clients fail at the same instant, they all compute the same base * 2 ** 1 = 2s and all retry two seconds later, together. You have moved the spike, not removed it.

Full jitter replaces the fixed delay with a random value between zero and that delay: Math.random() * ceiling. Here is 1,000 clients that all failed at once, bucketed by the 100ms window their retry lands in:

const noJitter   = Array.from({ length: 1000 }, () => 1000 * 2 ** 1);           // all identical
const fullJitter = Array.from({ length: 1000 }, () => Math.random() * 1000 * 2 ** 1);

no jitter   -> { '2000': 1000 }            all 1000 retries in one 100ms window
full jitter -> ~50 retries per window, spread across 20 windows (busiest ~63)

Without jitter, one 100ms window takes all 1,000 retries - an instant spike straight back onto the server. With full jitter the same retries spread evenly across two seconds. The variants come from AWS's well-known "exponential backoff and jitter" analysis; "full jitter" is the simplest and a fine default.

Only retry what might actually succeed

Retrying the wrong error is worse than not retrying. Retry network errors (connection refused or reset, DNS failure, timeout) and HTTP 429, 502, 503, and 504. Do not retry 400, 401, 403, 404, or 422 - the request is wrong or not allowed and will fail identically every time, just wasting your attempts.

function isRetryable(err) {
  const s = err.status;
  return err.code === 'ETIMEDOUT' || err.code === 'ECONNRESET' ||
         s === 429 || (s >= 500 && s <= 599);
}

Running the full helper against a service that returns three transient 503s and then succeeds, versus one that returns a 400:

Case 1: transient 503s
  attempt 0 failed (503); sleeping 50ms (ceiling 100ms)
  attempt 1 failed (503); sleeping 107ms (ceiling 200ms)
  attempt 2 failed (503); sleeping 390ms (ceiling 400ms)
  -> succeeded on call 4
Case 2: a 400
  -> gave up immediately after 1 call: 400 Bad Request

The 503s are retried with growing, jittered sleeps until the fourth call works. The 400 gets zero retries - isRetryable returns false, so the helper rethrows on the first failure.

One more rule: do not blindly retry a non-idempotent POST. If the first request actually succeeded but its response was lost, a retry creates a duplicate order or a double charge. Send an idempotency key so the server can recognise and ignore the repeat. And if the server sends a Retry-After header, obey it - it is telling you exactly when to come back.

Putting it together

The whole helper is about a dozen lines: loop over attempts, call the function, and on a retryable error sleep for a jittered exponential delay before the next try.

const sleep = (ms) => new Promise((r) => setTimeout(r, ms));

async function retry(fn, { retries = 5, base = 100, cap = 10_000 } = {}) {
  for (let attempt = 0; ; attempt++) {
    try {
      return await fn();
    } catch (err) {
      if (attempt >= retries || !isRetryable(err)) throw err;
      const ceiling = Math.min(cap, base * 2 ** attempt);
      const wait = Math.random() * ceiling;   // full jitter: uniform in [0, ceiling]
      await sleep(wait);
    }
  }
}

Prefer a library. fetch has no retry built in, but got, axios-retry, the AWS SDK, and the p-retry package all implement backoff with jitter correctly.

Add a retry budget. If more than roughly 10% of all your requests are retries, stop retrying and shed load - the system is in trouble and the retries are now part of the problem.

Do not stack retries at every layer. If the HTTP client retries, the service calling it retries, and the gateway retries, one user request can fan out into dozens of backend calls. Retry at one layer only - usually the outermost one that can still do something useful with the result - and log every retry, because a rising retry rate is an early warning that something is breaking.

Frequently Asked Questions