Reflection — The Agent That Reviews Its Own Work

Instead of one response, three steps: the model writes the draft. Then — the same model or another one playing critic — reads it with specific criteria in hand: what's wrong? What's missing? Finally the model rewrites based on that critique.

You already do this: before sending an important email, you read it again. The difference is that here we force the model to do that second read — with a checklist, not by mood.

You know the difference between code merged with no review and code that went through a proper code review: a reviewer looking at the diff, holding a checklist — security, performance, edge cases.

Reflection is the same idea inside the system: the draft is the PR, the critic is the reviewer, and the criteria are the checklist.

And exactly like real life: a reviewer without a checklist writes "LGTM" and moves on. That is the flattering critic — and we'll come back to it, because it's the most common way this pattern fails.

A real example

The task: "Get the top 5 customers by spend over the last 90 days."

First draft:

SELECT customer_id, SUM(total) AS spend
FROM orders
GROUP BY customer_id
ORDER BY spend DESC
LIMIT 5;

Looks fine, runs without error — and returns the wrong answer.

The critique (with criteria):

> - C1 — Time filter matches the request? ❌ No condition on the last 90 days at all. > - C2 — Cancelled and refunded orders excluded? ❌ It sums every status, so a customer who returned all their orders could rank as "top spender." > - C3 — Aggregation level matches the question? ✅ Grouped by customer — correct.

The revised version:

SELECT customer_id, SUM(total) AS spend
FROM orders
WHERE created_at >= CURRENT_DATE - INTERVAL '90 days'
  AND status NOT IN ('cancelled', 'refunded')
GROUP BY customer_id
ORDER BY spend DESC
LIMIT 5;

Same model. All that happened is someone asked it: "Are you sure? Look at the time window and the statuses." And note that both bugs are the kind that pass a quick smoke test — the query runs, the numbers look plausible, and the error hides in the meaning.

1. Produce: the model writes the draft normally. 2. Critique: a call in the "critic" role: take the original task + the draft + specific criteria, and produce a report: what fails and why. 3. Revise: a third call: the draft + the critique report → a revised version that addresses the listed points specifically.

The critic comes in two shapes — and choosing between them is an engineering decision, not a detail:

### Reflexion — when the critique gets memory and evidence

⚠️ Watch the name: Reflexion with an x — a proper name for a pattern from a research paper, not a typo.

Plain Reflection critiques within a single attempt and then forgets. Reflexion adds two things:

1. An objective external signal: instead of the critic's opinion alone — tests that pass or fail, an error from execution, a score from an eval. Something that doesn't flatter. 2. Memory of failure: after each failed attempt, the model writes a short summary ("Tried X, failed because Y — next time I'll try Z") that gets injected into the next attempt's prompt.

The result: attempt four doesn't start from zero — it starts from the lessons-learned file. This pattern crushes it on tasks with a checkable answer: code that must pass unit tests, a query whose result is compared against a known one, a problem with a correct solution. And with no external signal? Reflexion degrades to plain Reflection — just with a fancier name.

The single engineering point that separates working Reflection from decorative Reflection: the critic must be given an explicit rubric. A correct critic prompt looks like this:

You are a strict code reviewer. Evaluate the SQL below against
these criteria — for each one, output PASS or FAIL + one line why:

1. Time filter matches the request exactly
2. Cancelled and refunded orders are excluded
3. Aggregation level matches the question
4. Only existing columns are used (schema attached)

Then list the required fixes, most severe first.
Do NOT rewrite the query yourself.

Why the last line? Because without it the critic "escapes" from critiquing into authoring — writing you a new version to its own taste instead of a report you can build on. Separating the roles keeps each call specialized and inspectable.

Two more practical points:

• Make the critic's report structured output (JSON with PASS/FAIL per criterion) — so your code can stop the loop programmatically when everything passes, instead of parsing prose.
• The round cap lives in code, not in good intentions: MAX_ROUNDS = 2 from day one. The reasons are in the cost section.

💡 Tip: The rubric you wrote for the critic is the seed of your evaluation suite: these same criteria are what become a Golden Set and LLM-as-Judge in the AI Evaluation domain. BuildAI's philosophy is the same in both: without criteria, you're not evaluating — you're guessing. The only difference is the critic works inside the loop, and the eval judges from outside.

Exercise (20 minutes): Take a real summarization task — a page from your project docs, say. (1) Get the model's first draft and save it. (2) Write a 4-criterion rubric: factual accuracy, coverage of the key points, zero invented facts, length ≤ 150 words. (3) Run one Critique → Revise round with that rubric. (4) Show both versions to a colleague without saying which is which — and let them pick.

If the revised one wins: congratulations, you now know how to buy quality at 3×. If it didn't? You learned a more expensive lesson: review isn't free and isn't always worth it — and that measurement you just did was itself a small evaluation.

Cost considerations

One review round = three calls instead of one. Let's run it on a report-summarization task:

> 💸 Numbers subject to change *(Volatile fragment — illustrative pricing; check your provider's current rates)*: $3/million input tokens and $15/million output tokens.

At 5,000 summaries a month: ~$185 instead of ~$60. A second round? Add another ~$0.025 per summary — and here's the important news: most of the improvement comes from the first round. The second adds less, and the third usually shifts quality sideways rather than upward. Take it as a rule of thumb — not physics — but break it with measurement, not hope.

The decision in the end is simple: is the extra quality worth 3×? A report going to a client — yes. A quick chat reply — usually no.

• Output that gets handed to someone: reports, important emails, code up for review — anything where first impressions matter.
• Checkable tasks with tests or evals: go straight to Reflexion — external signal + failure memory.
• First-attempt quality isn't enough and switching models is off the table: review is often cheaper than a pricier model — do the math.
• Long-horizon agents: failure memory prevents repeating the same mistake within the same task.

• Fast replies where acceptable quality is enough: general chat, suggestions, internal drafts — 3× cost for marginal benefit.
• Latency-sensitive paths: three sequential calls ≈ three times the wait. On a hot interactive path, that's felt and painful.
• The problem is formal, not substantive: output missing a required format? Structured output + validation solves it in one call.
• All criteria are automatable and success is Pass/Fail: then you don't need a "review" — you need a quality gate. That's the next concept: Evaluator–Optimizer.