Large Language Models· Lesson 6 of 7
- 1How an LLM Works Inside8 min
- 2The Context Window — the Model's Short-Term Memory7 min
- 3Temperature & Sampling — Controlling Randomness7 min
- 4System vs User Messages — Who's Speaking7 min
- 5Choosing a Model — Bigger Isn't Always Better8 min
- 6What LLMs Can't Do7 min
- 7Cost & Latency — How Your Bill Adds Up7 min
What LLMs Can't Do
In one line · A model doesn't compute reliably, doesn't know events after training, and doesn't check a source — unless you give it those tools.
Most AI product disasters come from expecting capabilities that aren't there. Knowing the limits tells you when you need tools or RAG.
The model is like an expert speaking from memory: very fluent, but may state a wrong number or date with full confidence because it verifies nothing.
Like a slick improv actor who never says 'I don't know': whatever you ask, he'll finish the line with confidence and fill any gap from his head — even a gap that actually needed to be computed or looked up.
Ask 'how many days between two dates' and it gives a confident wrong number; give it a calculator tool and it's right. The limit is the model, the fix is design.
All these limits trace back to the same mechanism: the model generates the likeliest token from patterns it trained on. Arithmetic goes wrong because '7,234 × 891' is treated as a text pattern, not a computation. Recent events are absent because knowledge froze at the training-data cutoff. Citations get fabricated because the model generates the convincing shape of a reference, not a real one. The industry solves these limits by design: tool calling hands the model a calculator, search, and APIs, and RAG brings real documents into context before answering — turning the model from 'the source of facts' into 'a language engine operating over trusted information.'
🧪 Try it yourself — make the model fail a simple calculation: ask a model (without tools) to compute 7,234 x 891 and answer with just the number, no steps. Check with a calculator — it will usually be off by a digit or rounded, in a perfectly confident tone. Then ask the same in an interface with a code interpreter or calculator and compare. Repeat the idea with a question about an event after its training date, and with 'give me 3 academic references on a narrow topic' — then look the references up. You'll see all three limits with your own eyes.
- Believing an answer because the tone is confident. Model confidence is a stylistic property, not an accuracy signal — hallucination arrives in exactly the voice of truth.
- Shipping a product that relies on internal model knowledge for live data (prices, stock, news, company policy). Anything that changes must come from an external source in context.
- Treating mechanical limits with prompting. 'Please compute precisely' won't make the model a calculator — these limits are solved with tools and architecture, not better phrasing.
Turn this list into a design-review checklist: for every feature ask — where might it compute? Where does it need post-training information? Where will it claim a source? Every 'here' answer means a tool, RAG, or a human check in the path. This list is what determines the product's architecture.
Don't let the limits become blanket rejection — 'models hallucinate' is no reason to avoid them for summarizing, drafting, translation, and coding assistance, all tasks they excel at when the source is in front of them. And the list itself moves: today's models compute better than last year's, so re-check your assumptions periodically instead of building on last year's limits.