Retrieval-augmented generation (RAG) answers a question in two steps. First it retrieves the passages that are relevant from your own documents. Then a language model writes the answer using those passages, not only what it memorized in training.

Why RAG exists

A general model does not know your latest policy, your customer’s file, or a number that changed this morning. If you paste the whole library into every prompt, you exceed the context window and you bury the useful paragraph. RAG fetches a short, relevant set of chunks and asks the model to answer from that set.

The flow

1. Ingest documents: split them, embed the chunks, and store vectors plus the source text.
2. Retrieve: embed the question and pull the closest chunks. Filter by permission, product, or date first.
3. Augment: place those chunks in the prompt, with instructions to use only that evidence.
4. Generate: the model writes the answer and, ideally, cites the chunk it used.

RAG compared with the alternatives

Method	Changes the model?	Best when
Prompt only	No	The needed facts fit in the prompt
RAG	No	Facts live in documents that change
Fine-tuning	Yes	You need a new skill, format, or tone, not a lookup

A practical example

A banker asks, “What is the current fee for an early closure of a home loan?” RAG retrieves the fee schedule dated this quarter, and the model answers with that figure and the schedule name. Without retrieval, the model might quote a fee it saw in older public text.

What usually goes wrong

• The right passage is never retrieved, so the model answers from memory.
• Chunks are too large or too small, so the evidence is incomplete.
• The index is stale after a policy update.
• The prompt does not require a citation, so a wrong sentence is hard to audit.

What to remember

• RAG grounds an answer in documents you choose.
• Retrieval quality decides answer quality. A strong model cannot cite a passage it never saw.
• Use RAG for knowledge that changes. Use fine-tuning when you need the model to behave differently.