Timeouts & Idempotency — Requests That Don't Double-Fire

In one line · You set a time limit on a request (a timeout) so you don't wait forever, and you use an idempotency key so a retried request isn't executed twice.

Glossary:IdempotencyInferenceTrace

Why it matters

Models are sometimes slow, and without a timeout your product hangs; without idempotency a retry can do the same thing twice — send two emails or charge twice.

A timeout is like an iron's auto-shutoff: run too long and it cuts itself off. Idempotency is like an order number: resend the same number and the system knows it's the same order and won't repeat it.

An analogy

An idempotency key is like a receipt number at a restaurant: if the waiter vanished and you're not sure your order was heard, reordering with the same receipt number tells the kitchen it's the same order — so they don't cook it twice.

A request hangs 30 seconds, so the timeout cancels it — but note, cancelling tells you nothing about whether the server finished it; so you retry with the same idempotency key, and if the first one had gone through, the server returns the same result instead of doing it twice.

How it works

You put a timeout on the call (with AbortController, say) so a hung request gets cut instead of hanging your product. But here's the key point: the timeout only cuts your side — it tells you nothing about whether the server did the work; it may have finished after you gave up. That's why side-effectful requests get an idempotency key: a unique ID for the logical operation, sent in a header. When you retry with the same key, the server recognizes it — if the operation already ran, it returns the same stored result instead of executing again.

import { randomUUID } from "crypto";

const idempotencyKey = randomUUID(); // one key per OPERATION, not per attempt

async function safeCall() {
  const controller = new AbortController();
  const timer = setTimeout(() => controller.abort(), 30_000);
  try {
    return await fetch(url, {
      method: "POST",
      signal: controller.signal,
      headers: {
        Authorization: `Bearer ${process.env.API_KEY}`,
        "Idempotency-Key": idempotencyKey, // same key on every retry
      },
      body,
    });
  } finally {
    clearTimeout(timer);
  }
}

Practice

🧪 Try it yourself: build a mock endpoint that stores results by Idempotency-Key, and a client whose first attempt times out (make the server deliberately outlast the timeout) then retries with the same key. Verify the operation ran exactly once — then try a fresh key per retry and watch the disaster yourself.

Common mistakes

(1) Assuming a timeout means 'it didn't execute' and retrying a side-effectful operation without a key — so two emails go out or a charge lands twice. (2) Generating a fresh idempotency key on every retry — that defeats the whole point; the key belongs to the operation, not the attempt. (3) Setting a too-short timeout on model generation — long non-streaming replies routinely take 30+ seconds, and you end up cancelling healthy requests.

When to use it

A timeout on every network call, no exceptions; an idempotency key on any request whose retry could double a side effect: charging money, sending an email, writing to a database.

When NOT to use it

Pure reads (GETs and the like) don't need idempotency keys — repeating them is harmless, so don't add machinery for nothing. The timeout still applies there — the key doesn't.

LET'S BUILD ITDone reading — now build it yourself.Build a safe retry with an idempotency key

Test your understanding

1. A 'send the customer an email' request timed out. Retry right away?

2. A client generates a new idempotency key on every retry attempt. What happens?

3. Which request needs an idempotency key the most?

Related concepts

Errors & Retries — When the Call Fails Rate Limits — Your Ceiling per Minute Streaming — The Reply Arrives Word by Word

Was this clear?