Errors & Retries — When the Call Fails

In one line · APIs fail sometimes (network, load, transient glitches); the fix is telling retryable failures from permanent ones and retrying intelligently.

Glossary:BackoffInferenceJitterModelTrace

Why it matters

A product that dies on the first error won't survive production. Proper error handling is what turns a service that 'works' into one that's reliable.

Like calling a phone that doesn't answer: if it's busy you try again shortly (transient), but if the number's simply wrong there's no point redialing forever. Errors work the same way.

An analogy

Backoff with jitter is like a crowd stuck at a shut door: if everyone bangs on it at the same instant, it stays jammed. But if each person waits a slightly different, random spell before knocking again, the crush eases and the door opens.

A 429 or 503 (or a dropped connection) means 'try again shortly', so you retry with exponential backoff plus jitter — each attempt waits a bit longer plus a random amount so clients don't all retry in the same instant; but a 400 or 401 is a permanent error (bad key / malformed request) — retrying fixes nothing, you must fix the underlying problem.

How it works

Step one is classifying the error: 429, 503, and network drops are transient — retryable; 400 and 401 are permanent — retrying fixes nothing, you must fix the request or the key. Retries use exponential backoff: each attempt waits double the previous one plus random jitter so clients don't all come back in the same instant. If the server sends a Retry-After header you honor it and wait exactly that long, and you cap attempts (3-5) so a real outage fails cleanly instead of hanging.

async function callWithRetry(fn, maxAttempts = 4) {
  for (let attempt = 0; ; attempt++) {
    try {
      return await fn();
    } catch (err) {
      const transient =
        err.status === 429 || err.status === 503 || err.code === "ECONNRESET";
      if (!transient || attempt >= maxAttempts - 1) throw err; // 400/401: fail fast
      const retryAfterMs = Number(err.headers?.["retry-after"]) * 1000 || 0;
      const backoffMs = 500 * 2 ** attempt + Math.random() * 300; // jitter
      await sleep(Math.max(retryAfterMs, backoffMs));
    }
  }
}

Practice

🧪 Try it yourself: write a callWithRetry function that retries only on 429/503/network drops, with exponential backoff plus jitter and a 4-attempt cap, honoring Retry-After when present. Test it against a mock server that returns 429 twice then 200 — and one that returns 401 to confirm it fails immediately.

Common mistakes

(1) Retrying a 401 in a loop — burning time and requests on an error that can never succeed; the key is wrong, fix it. (2) A fixed delay with no jitter — every client returns at the same instant, creating a fresh stampede (thundering herd). (3) Ignoring Retry-After and guessing your own delay — the server told you exactly how long to wait.

When to use it

Every production network call to an AI API — write one retry helper and use it everywhere instead of duplicating the logic.

When NOT to use it

Never retry permanent errors (400/401/403), and never blind-retry a request with a side effect (a charge, an email) without an idempotency key — it can execute twice (see the timeouts & idempotency lesson).

LET'S BUILD ITDone reading — now build it yourself.Build a retry with exponential backoff

Test your understanding

1. Which errors deserve a retry?

2. 1000 clients failed together, and all will retry after exactly one second. What happens?

3. You get a 429 with a `Retry-After: 12` header. What do you do?

Related concepts

The First Call — Reaching the Model From Code Rate Limits — Your Ceiling per Minute Timeouts & Idempotency — Requests That Don't Double-Fire

Was this clear?