Retrieval is a library card.

Retrieval-augmented generation (external link) (Lewis et al., 2020) named a simple pattern: fetch documents, then generate. You index a corpus you approve. At ask-time you retrieve a few chunks. Those chunks go into the working context so the LLM can ground instead of inventing a policy from 2021 training data.

That is RAG. It is not an agent. It is sort of like a search box that summarises quotes from a prompt. For enterprise depth — ops, product instances, and when to choose RAG over an agent — see RAG Explained.

Index, retrieve k, then generate

Lewis et al. are specific: you index a corpus, retrieve a small set of passages, then condition generation on those passages. In operations that looks like:

corpus  →  index (embeddings or search)
query   →  top-k chunks
chunks + question  →  LLM

k is a budget. Too small: you miss the policy paragraph. Too large: you crowd the window that memory already fights for, and the model may ignore the middle of a long paste. Liu et al. document that lost-in-the-middle failure in arXiv:2307.03172 (external link): models often use the start and end of a long context better than the buried middle. That is why “stuff the whole wiki” is not RAG. RAG is retrieve k, then generate.

Retrieve, then ground

The mechanism:

  1. Turn the need into a query (sometimes the user sentence, sometimes a rewrite).
  2. Retrieve from your store — wiki, tickets, contracts — not “the internet” unless that is an explicit tool.
  3. Stuff only the useful snippets into context.
  4. Generate the next thought, answer, or function call.

Stale indexes are silent failures. Wrong chunk, confident answer. Duplicate chunks waste the window that memory already fights for. Citations in the UI help humans; they do not magically make the model honest if the chunk was about a different product.

When retrieval becomes agentic

From Models to Agents Part 10 already named agentic RAG: retrieve from an approved knowledge base, then act only within those limits: open the ticket the doc describes, fill the form the policy allows. This part teaches the split so that name has a diagram.

Retrieve  →  Ground (write)     still RAG
Retrieve  →  Ground  →  Tool    agentic RAG

To safeguard data, retrieval needs ACLs (Access Control Lists). Retrieval plus tools gives an agent access to your documents, like a library card. Those documents still have to be yours and up to date. Part 10 covers the name for this pattern. This part focuses on the retrieve-then-ground pipeline.

Example

Policy PDF says refunds over $50 need a manager approval. RAG pulls that paragraph. Grounded chat: the model tells you. Agentic: it drafts the ticket and stops before refund because the retrieved rule and the tool catalogue agree. If retrieval missed the PDF, the agent may still try to be “helpful” (in a bad way).

Design that doesn’t depend on perfect retrieval

  • Hard policy in the runtime, soft policy in RAG
    Encode “refunds over $50 need approval” as something the code enforces: no refund tool, or refund requires an approval_id the runtime checks. RAG explains why; the allowlist decides whether. That matches agent-policy: prompt guides, policy enforces.

  • Least privilege by default
    Prefer draft_ticket / request_approval over refund. If retrieval fails, the worst case is a draft, not a payment.

  • Mandatory retrieve-before-irreversible
    Orchestration rule: before any write tool in a class (refund, send, delete), require a successful retrieval (or a pinned policy doc ID) and log the chunk IDs. No cite → no call; escalate to human.

  • Pin critical policies, don’t only search them
    Put high-risk rules in the system prompt or a versioned “policy pack” always injected for that agent job. Use RAG for long-tail docs; use a short allowlist of must-load policies for money/safety.

  • Eval the miss as a first-class case
    As the post says: fixture where the PDF isn’t retrieved (or wrong product chunk is). Pass = stops / asks / drafts only. Fail = refund called. That’s how you make the design honest, not fail-proof.

One-line rule: RAG lets the agent read approved documents, while actions that cannot be undone stay locked behind approval. Never let the only approval rule sit inside a PDF that search might miss.

Conclusion

Put RAG on the diagram as a tool-shaped read against a corpus, feeding working context. Then decide whether the next step is prose or a function call. That decision is orchestration.

Takeaway: RAG is retrieve-then-ground from an approved corpus. Agentic RAG adds a tool after the quote — it is not a different kind of language model.

Sources