RAG is not obsolete. The dated pattern is one query, one vector search, a fixed top-k, and an answer whether the evidence is sufficient or not. Agentic RAG adds a controlled retrieval loop: plan what evidence is needed, choose the right source and search method, inspect results, search again when gaps remain, and stop when the answer is supported or the budget is exhausted.
Google Research's June 2026 work on Agentic RAG frames retrieval as an active reasoning process rather than a single preprocessing step. That is useful for messy company knowledge—but only when the task genuinely needs multiple searches or tools.
Classic RAG is still the right baseline
Start with a conventional permission-aware retrieval pipeline when questions are direct, the corpus is coherent, and one search usually finds the answer.
A strong baseline already includes:
- authoritative sources and stable source identifiers;
- structure-aware parsing and metadata;
- semantic, lexical, and filtered retrieval;
- reranking;
- precise citations;
- no-answer behavior;
- update, deletion, and access-control handling;
- a maintained evaluation set.
Do not add an agent loop to compensate for weak parsing, missing metadata, or an unmeasured baseline. More searches over a poor index create more expensive confusion.
When iterative retrieval earns its cost
Agentic retrieval is useful when a question must be decomposed, evidence lives in different systems, the first search is predictably incomplete, or the system needs to compare and reconcile sources.
Examples:
- “Which customer contracts renew next quarter, and what exceptions appear in their latest correspondence?”
- “Why did this invoice fail validation, and which policy and purchase-order fields apply?”
- “Compare the current product policy with the procedure described in these training videos.”
- “Prepare a supported answer using internal documents and current public sources.”
These tasks need several evidence operations, not a larger top-k.
The retrieval loop
1. Define the evidence target
Translate the request into claims or subquestions that need support. Mark which source types are authoritative for each one. An invoice database may be authoritative for status; a policy library for rules; email for exceptions; public search for current external facts.
2. Route to the right retrieval method
Not every source belongs in one vector index. Use:
- keyword search for names, codes, exact phrases, and identifiers;
- semantic search for conceptual similarity;
- metadata or SQL filters for dates, tenants, statuses, and ownership;
- graph traversal for relationship-heavy questions;
- visual or layout-aware retrieval for tables, diagrams, and scanned pages;
- direct application APIs for current operational state;
- web search only when external evidence is allowed and needed.
The agent chooses among approved retrieval capabilities; it does not invent access.
3. Inspect evidence, not just scores
The system should evaluate whether the returned passages actually address the subquestion, come from an appropriate source, are current, and agree with one another. High similarity is not proof.
Google's sufficient-context research shows why this matters: a model's ability to answer depends on whether the retrieved context contains enough evidence, not merely whether it looks relevant. See Sufficient Context: A New Lens on Retrieval Augmented Generation Systems.
4. Search again with a reason
If evidence is missing, rewrite the query, follow an entity or citation, change retrieval mode, narrow a filter, or ask the user for a key detail. Every additional search should correspond to a named gap.
Avoid an unbounded “research until confident” loop. Track searches, elapsed time, tokens, source calls, and the claims still unsupported.
5. Stop, synthesize, or abstain
Stop when each material claim has adequate evidence, the task's budget is reached, or further searches are repeating the same information. If support remains insufficient, state what was checked and what is missing. For high-risk work, route the case to a person.
Hybrid retrieval and reranking still matter
Agentic behavior does not replace retrieval engineering. Exact identifiers often need lexical search. Dense embeddings help with meaning. Metadata filters enforce scope. Late-interaction or cross-encoder rerankers can improve candidate ordering after broad retrieval.
Choose a simple measured combination before adopting more elaborate infrastructure. Evaluate by query category: entity lookup, policy explanation, multi-hop reasoning, conflict detection, table extraction, and unanswerable requests can need different retrieval paths.
GraphRAG is a tool for specific query shapes
Graph-based retrieval can help when the answer depends on entities and relationships spread across many documents—for example, people, organizations, events, dependencies, or themes across a large corpus. Microsoft's GraphRAG separates local entity-focused and broader corpus-level search patterns.
It is not a universal upgrade for document Q&A. Graph extraction, indexing, and updates add cost. Use it when evaluation shows that relationship or whole-corpus questions repeatedly fail with simpler retrieval.
Permissions must constrain every iteration
The agent must never retrieve broadly and rely on the model to hide unauthorized material. Resolve the authenticated user and tenant in trusted application code, then apply source and record permissions to every search, follow-up, preview, citation, cache, and export.
Test indirect paths as well:
- a follow-up query generated from restricted content;
- an entity link that crosses tenant boundaries;
- cached evidence from a previous user;
- a document whose access changed after indexing;
- a citation URL copied into another session.
The search plan is model-generated; the authorization boundary is not.
Keep an evidence ledger
For complex answers, record each material claim alongside its source ID, version, location, retrieval step, and support status. This makes it possible to:
- generate precise citations;
- reveal conflicts instead of blending them;
- remove claims whose evidence disappeared;
- inspect why a search loop continued;
- evaluate claim-to-source correctness;
- reproduce the answer after a model or index change.
For video, citations may need timestamps. For PDFs, page and section. For structured records, an object ID and relevant fields. Makhfi AI worked across text and video-derived knowledge, so source identity and the path back to original context were part of the product problem, not decorative links.
Evaluate the loop, not just the answer
A useful suite measures:
- whether the search plan covered the required subquestions;
- source routing and permission correctness;
- retrieval relevance and evidence coverage;
- unnecessary or repeated searches;
- conflict identification;
- citation entailment and precision;
- supported abstention;
- task completion;
- latency and cost by step.
Include cases where one-shot retrieval is sufficient. A good agentic system should not turn a direct lookup into six model calls.
A production decision checklist
- [ ] Do real questions require decomposition or multiple sources?
- [ ] Is a measured classic-RAG baseline already failing those questions?
- [ ] Are source authority and permissions explicit?
- [ ] Can the agent select among narrow, observable retrieval tools?
- [ ] Does every follow-up search name the evidence gap it addresses?
- [ ] Are stop conditions and budgets enforced outside the prompt?
- [ ] Can every material claim be traced to a versioned source?
- [ ] Does the suite test conflicts, missing evidence, and access changes?
Agentic RAG is best understood as adaptive evidence gathering, not a marketing replacement for retrieval. Use it where the question needs a research loop; keep simple questions simple.