Large Language Models· Lesson 5 of 7
- 1How an LLM Works Inside8 min
- 2The Context Window — the Model's Short-Term Memory7 min
- 3Temperature & Sampling — Controlling Randomness7 min
- 4System vs User Messages — Who's Speaking7 min
- 5Choosing a Model — Bigger Isn't Always Better8 min
- 6What LLMs Can't Do7 min
- 7Cost & Latency — How Your Bill Adds Up7 min
Choosing a Model — Bigger Isn't Always Better
In one line · Model choice is a trade-off of intelligence, cost, and speed — the right model is the cheapest one that passes your task.
Using the strongest model for everything burns money and slows your product with no real gain on simple tasks.
Like picking a vehicle: you don't take a truck to buy bread. The task sizes the model, not the other way round.
Like hiring: you don't put a highly paid senior engineer on resetting a password. You staff to the task, and reach for the expensive hire only when the work justifies it.
Classifying a message as 'complaint/inquiry' a small cheap model nails; a complex report or multi-step reasoning needs a stronger one. Try the smaller first and move up only if it fails.
Every provider offers a 'ladder' of models: small/fast/cheap → medium → large/expensive, sometimes plus reasoning models that spend extra thinking time on hard problems. The practical differences show up on 4 axes: multi-step reasoning quality, cost per million tokens, latency, and context window size. The sound method is the opposite of instinct: start with the cheapest that could plausibly pass, build a small test set (evals) from real examples of your task, and measure. The model that passes your evals at the lowest cost is the 'right' one — not the top scorer on public benchmarks. Mature products add routing: simple tasks go to the small model, and only complex ones escalate to the big one.
🧪 Try it yourself — run one task on two models: pick a real task from your work (say: classifying 20 customer messages into complaint/inquiry/request). Prepare the 20 examples with correct labels, then run them through the provider's smallest model and its largest. Compute per model: accuracy, average latency, and the approximate bill if this task ran a million times a month. The usual surprise: on simple classification the quality gap is near zero and the bill gap is enormous.
- Choosing by public benchmarks instead of your task. A model that tops math benchmarks can be mediocre at Arabic summarization in your domain — nothing replaces an eval on your data.
- Starting with the priciest 'to be safe' and never revisiting. Small-to-large price gaps run 20–60x — at a million requests a month, that bill makes or breaks the product.
- Forgetting latency. Stronger models are usually slower; in an interactive UX, two extra seconds can hurt more than a slightly less brilliant answer.
Make a deliberate choice (with a small eval) for any feature that calls the model at volume or frequency: savings multiply with scale. And revisit the decision every few months — the market moves fast, and today's small model does last year's big model's job.
Don't spend a week optimizing model choice for a feature called 100 times a day — the cost delta is pennies and any mid-tier model will do. And at the prototype stage, start with a strong model to prove the idea works at all, then walk down the ladder — optimizing early confuses 'the idea doesn't work' with 'the model is too weak.'