RAG (retrieval-augmented generation)
The architecture that retrieves documents at query time and has a model write from them, and the reason your page can be quoted without ever being trained on.
Karl-Gustav Kallasmaa, Founder & CEOLast updated Retrieval-augmented generation is the arrangement in which a system fetches relevant documents at query time and gives them to a language model, which then writes its answer from what it just read. It is the architecture behind almost every AI product that can show you a working link alongside its answer.
The term comes from a 2020 paper, Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks, by Patrick Lewis and colleagues, first submitted in May 2020 and last revised in April 2021. It described models that combine pre-trained parametric and non-parametric memory for language generation: a pre-trained sequence-to-sequence model as the parametric part, and a dense vector index of Wikipedia accessed with a neural retriever as the non-parametric part. The paper reported state-of-the-art results on three open-domain question answering tasks, and that its models generated more specific, diverse and factual language than a parametric-only baseline.
Anthropic's glossary gives the same idea in operational terms: the model is augmented with an external knowledge base that is passed into the context window, retrieved at runtime when a query is sent, which lets the model use information beyond its training data and reduces reliance on memorisation.
Why it is the important architecture for publishers
Because it is the only one in which what you publish today can affect an answer today.
A purely parametric answer draws on training data fixed months earlier. Nothing you write now reaches it, and any URL it produces is reconstructed from memory rather than fetched. A retrieval-based answer fetches live documents, which means three things at once: your current page is eligible, the answer can carry a real citation, and an error you fix is fixed in the next answer rather than in the next model.
It also means access is upstream of everything. A retrieval step is an HTTP fetch by a named agent, so a page the agent may not fetch is not a candidate at all, however well written. AI search covers which agents do this job, and robots.txt covers how you answer them.
The pipeline, and where each stage fails
A typical implementation chunks a corpus, embeds each chunk, indexes the vectors, retrieves the nearest ones for a query, and passes the highest-scoring ones into the context window with the question. Each stage has its own failure mode, and they are usually diagnosed as one:
- Chunking. A chunk covering four topics has a vector that means none of them. This is a writing problem as much as an engineering one.
- Retrieval. Vector similarity is good at paraphrase and poor at exact identifiers, so a part number or a code can be missed while something that merely looks similar is returned.
- Selection. Only some retrieved passages fit the budget. Anthropic documents a 1M-token context window on several current Claude models — large, and still finite against a corpus that is not, which is why selection cannot be skipped.
- Generation. The model can write confidently from a passage that answers a different question. A citation records what was retrieved, not that it was correct.
That last distinction is worth holding onto: retrieval failures and generation failures look identical from outside, and only the first is something a publisher can influence.
How to act on it
- Write for passage selection. The retrieved unit is a section, not a page. Give each section one subject and a heading that states its claim.
- Repeat the qualifier. A caveat two paragraphs above the claim does not travel with the claim.
- Keep identifiers literal. Product codes, version numbers and names should appear as text a lexical matcher can find, not implied.
- Make the page cheap to fetch. An unauthenticated, fast, server-rendered page is a candidate; a slow one that renders client-side may time out of the pool. See crawling and indexing.
- Verify what is being said about you by sampling answers, since a wrong claim sourced to your page is a retrieval-and-generation failure you can often fix by editing the passage that caused it.
Frequently asked questions
Where does the term come from?
A 2020 paper by Lewis and colleagues combining parametric and non-parametric memory for generation.
How does it differ from training on my content?
It fetches documents while answering, so the sources exist, can be cited, and can be edited.
Do bigger context windows make it unnecessary?
No. The window is finite and accuracy degrades as it fills.
Why does a cited page sometimes not support the claim?
Because retrieval and generation fail separately, and a citation records only what was retrieved.
Terms related to RAG (retrieval-augmented generation)
Search where a model composes the answer and fetches pages through named crawlers, rather than returning a ranked list of links for you to read.
Vectors of floating point numbers whose distance measures relatedness, the representation underneath semantic search and retrieval.
How an AI answer attributes what it says to the pages it read, and why a citation is a distinct outcome from a click or a mention.
The token budget a model can reference in one request, what counts against it, and why a bigger window does not remove the need to retrieve selectively.