RAG· Lesson 1 of 4
What Is RAG?
In one line · RAG retrieves the right documents first, then has the model answer from them with citations — instead of improvising from memory.
RAG (Retrieval-Augmented Generation) means: retrieve → ground → cite. You search your data, put what you found into the prompt, and the model answers from that text only. The result: answers grounded in your actual sources, not guesswork.
The LLM doesn't know your private documents, and when you ask about them it hallucinates confidently. RAG is the standard fix — and the most-built enterprise AI pattern today. If you're building anything on private data, you'll probably build it with RAG.
Instead of asking the model 'what's the vacation policy?' and hoping it remembers, you first search your company files, pull the paragraph with the policy, and tell the model: 'answer from this text only.' Now the answer comes from your source, not its memory.
Like an open-book exam: the student isn't expected to memorize the book, just to open the right page, answer from it, and say which page. Retrieval is opening the page, grounding is answering from it, citation is the page number.
A user asks 'do we refund shipping?'. The system searches the support docs, finds the returns-policy clause, puts it in the prompt, and the model replies: 'Yes, within 14 days — source: Returns Policy, section 3.'
You convert your documents into embeddings and store them in a vector store. When a question comes in, you embed it too, fetch the closest chunks, put them in the prompt with 'answer from this context and cite the source,' and the model generates the answer.
The full pipeline: chunking → embedding → indexing at setup time, then retrieval (usually similarity search plus reranking) → prompt assembly → generation at question time. RAG quality is governed by retrieval quality more than model quality — wrong chunks mean wrong answers no matter how strong the model is.
🧪 Try it: take 3 paragraphs from any document you have, paste one into the prompt with a question about it and the instruction 'answer only from this text; if the answer isn't there, say I don't know.' Then ask a question whose answer isn't in the paragraph and see whether it complies or improvises. That's manual RAG — real systems just automate the 'pick the paragraph' step.
The most common mistake: believing RAG eliminates hallucination entirely. It slashes it on your data, but if retrieval returns the wrong text or you don't instruct 'say I don't know if it's not in the context,' the model still improvises. Second mistake: confusing RAG with fine-tuning — they solve different problems.
Use RAG when answers must come from private or changing knowledge (company docs, policies, product documentation), when you need citations, and when the data is bigger than any context window.
Don't use RAG to change the model's style or behavior — that's fine-tuning's job. And if your knowledge is small, static, and fits in the context window, just put it in the prompt and skip the pipeline complexity.