RAG and LLMs are not alternatives — here's how they relate

The short answer

An LLM (large language model) is a model. RAG (retrieval-augmented generation) is a method for using one. An LLM generates text from what it learned during training. RAG adds a retrieval layer in front of the LLM that finds relevant documents and places them in the prompt. RAG is not a model, and it cannot work without one, because the LLM is still the part that writes the answer.

Side-by-side comparison

An LLM on its own relies on parametric knowledge, meaning whatever is stored in its weights from training. That knowledge stops at a cutoff date and covers only the data the model was trained on. It cannot see your private files and it may hallucinate when asked about things it does not know.

A RAG system uses the same LLM but also searches an external knowledge base for each question. The model sees fresh and private information in its prompt, and the answer can be traced back to the passages that were retrieved. The LLM supplies language ability and reasoning. The retrieval layer supplies facts.

What each part is made of

An LLM is a single trained neural network, usually served behind an API or an inference endpoint. A RAG system is a pipeline of several parts: a data ingestion process, a chunking step, an embedding model, a vector database, a retriever, an optional re-ranker and finally the LLM. The LLM is one component among many.

RAG versus fine-tuning

People often compare RAG with fine-tuning, which is a different way of changing LLM behaviour. Fine-tuning updates the model's weights to teach it a style, format or skill, and it is costly to repeat. RAG leaves the weights alone and changes only the prompt, so new data is available as soon as it is stored in the vector database. RAG avoids the need for constant fine-tuning just to reach new data.

The two approaches also combine well. Its example project fine-tunes a model for a particular writing voice and uses RAG to supply the content it writes about.

Which should you use?

Use a plain LLM when the task depends on general knowledge or language skill, such as brainstorming or rewriting text you provide. Add RAG when answers must rely on private, recent or frequently changing information, or when you need answers you can verify against sources. For the details of what a RAG pipeline contains, read our guide to the components of a RAG pipeline.