What is RAG?

Large language models are impressive, but they have a fundamental limitation: they can only work with what they learned during training. Ask about your company's internal policies or last week's sales numbers, and they'll either make something up or admit they don't know.

Retrieval-Augmented Generation (RAG) solves this by adding a retrieval step before generation. When you ask a question, the system first searches a knowledge base (your documents, databases, wikis) for relevant information, then feeds that context to the LLM along with your question. The model generates its answer based on the retrieved facts, not just its training data.

The typical RAG pipeline works like this: documents are split into chunks and converted into vector embeddings (numerical representations of meaning). When a query comes in, it's also converted to an embedding, and the system finds the most semantically similar chunks. Those chunks become the context for the LLM's response.

RAG is the most practical way to make AI useful for real business tasks right now. Instead of fine-tuning a model (expensive, slow, requires ML expertise), you can point a RAG system at your existing documents and get accurate, source-backed answers within days.

It's also the foundation for most enterprise AI products you see today — customer support bots that know your product docs, internal search tools that understand your wiki, and coding assistants that know your codebase.