Agentic RAG is Retrieval-Augmented Generation where the agent decides what to retrieve, when to retrieve, and how to use the retrieved information.
In traditional RAG, the retrieval is a fixed pipeline: every query goes through the same retrieval step, retrieves the same kind of documents, and feeds the results into the same prompt. In agentic RAG, the agent has the retrieval as a tool: the agent decides whether to retrieve, what to retrieve, how to query, and how to use the results. The agent's policy is the policy that decides when retrieval is needed.
Why agentic RAG is the right pattern
The traditional RAG pipeline is right for a small, well-defined corpus where the queries are predictable. The agentic RAG pattern is right for larger, more complex corpora where the queries are varied and the agent needs to adapt the retrieval strategy to the query.
The agentic RAG pattern is also right when the agent's retrieval strategy needs to evolve over time. The agent's policy can be updated as the operator learns which retrieval strategies work better, and the operator can update the policy without rewriting the retrieval pipeline.
The agentic RAG workflow
The agentic RAG workflow typically looks like:
1. Decide. The agent decides whether retrieval is needed. For some queries, the answer is in the agent's working memory; for other queries, the agent needs to retrieve.
2. Query. The agent constructs a query. The query is not necessarily the user's original query; the query is the agent's translation of the user's intent into a retrieval query.
3. Retrieve. The agent invokes the retrieval tool (typically a vector search over a embedding index, or a keyword search over a text index).
4. Filter. The agent filters the retrieved documents. The agent decides which documents are relevant and which are not.
5. Synthesize. The agent synthesizes the filtered documents into a response. The agent's synthesis is the final answer.
6. Reflect. The agent reflects on the response. The agent decides whether the response is good enough, and the agent may decide to re-retrieve with a different query.
Operator implications
The agentic RAG pattern is the right place to start when an operator is adding retrieval to an agent that is already running. The pattern is also the right place to start when an operator is debugging an agent that is producing retrieval-heavy responses but not making progress on the user's intent. The most common operator issues with agentic RAG are: the agent is over-retrieving (the agent retrieves even when the answer is in the working memory), the agent is under-retrieving (the agent does not retrieve when it should), the agent's queries are too narrow (the agent misses relevant documents), and the agent's queries are too broad (the agent retrieves too many irrelevant documents).
Related terms
Agentic RAG is the use of embedding search or graph-based memory within an agent loop. The retrieval step is exposed to the agent as a tool. The retrieved documents become part of the memory layer for the current session. The retrieval's quality is one of the inputs to the observability layer.
For the full primer, see From Prompts to Agents.