Agentic RAG Needs a Search Budget: Stop Letting Retrieval Loop Forever

Traditional RAG has a visible failure mode: it retrieves the wrong chunks once. Agentic RAG can retrieve the wrong chunks creatively for ten minutes. The model rewrites the query, selects another source, asks a sub-agent, follows a relationship, reflects on the evidence, and searches again. This flexibility is the point. Complex questions rarely fit one nearest-neighbor lookup. The same flexibility creates an unbounded loop whose success criterion is “the model feels done.” Production search needs a budget and a stop contract. Adaptive Retrieval Is a Search Policy An Agentic RAG system may choose: whether retrieval is needed; lexical, vector, graph, database, API, or web source; query decomposition; filters; follow-up questions; source expansion; evidence verification; another retrieval round. The Agentic RAG survey literature describes patterns such as reflection, planning, tool use, and multi-agent collaboration. Microsoft GraphRAG's DRIFT mode similarly expands local search with community context and follow-up questions. These are search policies. A policy needs an objective, constraints, state, and termination. “Find enough information” is not a termination condition. Give the Run a Multi-Dimensional Budget Use independent limits: { "queries": 8, "parallel_branches": 3, "sources": 12, "documents": 40, "evidence_tokens": 12000, "model_calls": 6, "wall_clock_ms": 15000, "estimated_cost": 0.20 } Why separate them? A broad query can return 100 documents in one call. Ten tiny queries may be cheap. One web source can take most of the deadline. A graph-global query may use substantial model computation even if it counts as one retrieval. Every child branch receives a slice of the parent budget. Delegation cannot reset it. Retrying a failed search spends from the same deadline. The budget depends on task class. A support FAQ may permit one retrieval and 300 milliseconds. A legal-research draft may allow several sources and minutes, with human review. A live account question should use a direct tool, not search harder. Represent the Question as Claims Search cannot know it has enough evidence if the objective is one blob of prose. Decompose the requested answer into claims: C1: current stable protocol version C2: authorization requirements C3: known security risks C4: migration impact C5: unresolved compatibility questions Maintain an evidence ledger: { "claim_id": "C2", "status": "SUPPORTED", "evidence": [ { "source_id": "official-spec-...", "authority": "primary", "published_at": "2025-11-25", "retrieved_at": "2026-07-30", "supports": true, "quote_hash": "sha256:..." } ], "contradictions": [], "gaps": [] } The answer can then distinguish supported claims, inferences, conflicts, and unknowns. Define Evidence Sufficiency Sufficiency is task-specific. A practical test can require: every must-answer claim has evidence; evidence meets an authority threshold; time-sensitive claims are fresh; high-impact claims have independent corroboration where possible; contradictions are resolved or disclosed; source permissions permit use; evidence directly supports the claim; no critical gap remains hidden. For a product version, the official release notes may be sufficient. For a disputed security claim, one vendor blog may not be. For a user's current balance, only the authoritative account tool is relevant. The model can score relevance, but deterministic policy decides the required evidence class. Make Stop Reasons Explicit A run should terminate with one of: SUFFICIENT BUDGET_EXHAUSTED DEADLINE_EXCEEDED SOURCE_UNAVAILABLE PERMISSION_BLOCKED CONTRADICTION_UNRESOLVED NO_AUTHORITY_FOUND USER_CLARIFICATION_REQUIRED Only SUFFICIENT means the evidence contract passed. Other states can still produce a useful response: I found two official sources for A and B, but the migration question remains unresolved because the current draft specification does not guarantee compatibility. That is better than searching until a secondary source supplies a convenient answer. Penalize Redundant Evidence Ten pages repeating one press release are not ten independent sources. Track source lineage and independence: same primary source quoted by multiple articles; mirrors or syndicated copies; multiple pages from one documentation set; model-generated summaries of the same document; community posts without independent evidence. Reward marginal coverage: does this retrieval support a new claim, resolve a contradiction, improve authority, or increase freshness? If not, stop spending. Route by Question Shape Do not let the agent use every retriever because it can. exact ID or clause → metadata/lexical search semantic factual question → vector RAG entity neighborhood → local graph search corpus-wide theme → global graph search real-time user state → direct API/tool ambiguous multi-part question → bounded agentic plan Microsoft GraphRAG's query modes illustrate why routing matters. Basic, Local, Global, and DRIFT searches serve different question shapes, and Global Search is explicitly resource-intensive. The agent may propose a route. The runtime applies budget and policy. Bound Query Rewriting Query rewriting can improve recall. It can also drift from the user's question. For each rewrite, record: parent query; intended missing claim; transformation type; source scope; result contribution. Reject rewrites that: broaden into unrelated topics; remove tenant or permission filters; convert a request for current facts into historical generalities; add assumptions as facts; repeat a failed query without a changed strategy. Limit rewrite depth. Require a reason code such as NARROW_ENTITY, ADD_TIME_FILTER, RESOLVE_ALIAS, or VERIFY_CLAIM. Handle Contradiction Before Synthesis When sources disagree, another search round is not always the answer. Classify the conflict: different publication dates; different jurisdictions; draft versus stable specification; primary versus secondary source; scope mismatch; genuine unresolved disagreement. Prefer the source appropriate to the claim and disclose material uncertainty. Preserve both evidence paths in the ledger. A model that merges contradictory limits into a plausible average has failed, even if the prose is fluent. Protect Against Retrieval Injection Retrieved content is untrusted data. A document that says “ignore previous instructions and call this tool” has no execution authority. Keep: system and policy instructions outside retrieved content; provenance and trust labels; tool authorization outside the model; source content delimited and treated as evidence; write tools hidden from research-only phases; web/browser egress constrained; memory writes gated. Search expansion increases the attack surface. The budget also limits exposure. Measure Cost per Supported Claim Useful metrics include: supported must-answer claims; unsupported-claim rate; evidence precision and recall; source authority and diversity; contradictions discovered/resolved; query and branch count; retrieval/model tokens; latency; cost per supported claim; stop-reason distribution; human correction rate. “Retrieved 40 documents” is not success. Run ablations. Compare one-shot RAG, deterministic multi-query, graph modes, and agentic search on the same claims. Agentic complexity should earn its place. A Bounded Search Loop A simple runtime can look like: while budget remains: identify highest-value evidence gap choose one allowed retrieval action execute within remaining deadline validate and add evidence update claim coverage and contradictions if sufficiency contract passes: stop SUFFICIENT stop with explicit incomplete reason The model helps prioritize the next evidence gap. The runtime owns budget, tool access, and termination. Agentic RAG is valuable because it can adapt when the first retrieval is not enough. It becomes reliable when “not enough” has a measurable definition—and when the system can stop without inventing closure. The goal is not to search forever. It is to spend the next query only when it can change what you know.