How to Build Agents

Agent Design Patterns

Top 5 Agentic RAG Patterns for Retrieval-Augmented Generation

Agentic RAG can mean anything from letting a model choose its next retrieval tool to building a graph-based workflow that synthesizes information across an entire corpus. The right pattern depends less on which framework sounds most sophisticated and more on what your queries ask, how your data is organized, and what controls you need around retrieval and failures.

This guide compares five documented approaches: LangGraph, Haystack, LlamaIndex, Microsoft GraphRAG, and Neo4j GraphRAG. I’ll focus on when each is useful, what it lets you control, and where extra implementation work or resource use may matter.

Treat these as architectural options, not a universal ranking or a guarantee of answer accuracy.

Digital illustration of cyberpunk cats in neon-lit urban scenes, engaged in AI-agent tasks such as coding, debugging, or analyzing data streams. The style features glowing outlines, high contrast, and a vibrant acid-color palette. No brand logos, no invented software screenshots, no legible text.: Top 5 Agentic RAG Patterns for Retrieval-Augmented Generation

Quick picks

For a quick starting point, match the pattern to the work your RAG system needs to do:

  • LangGraph — custom, stateful workflows: Use explicit nodes and conditional routes when later retrieval or review steps depend on intermediate results. You’ll need to design the workflow and its controls.
  • Haystack — bounded tool-calling loops: Consider it when an agent should iteratively choose retrieval tools, with configured exit conditions and a maximum-step limit. Error handling is not factual verification.
  • LlamaIndex — query-based retriever routing: A RouterRetriever selects among the candidate retrievers you provide, making it useful when queries call for different retrieval strategies. It does not establish that generated answers are correct.
  • Microsoft GraphRAG — corpus-wide synthesis: Global search uses community reports and map-reduce processing for questions about themes or the dataset as a whole. It requires an index and can use more time and LLM resources at detailed hierarchy levels.
  • Neo4j GraphRAG — mixed graph and document retrieval: Consider its vector, hybrid, Cypher, and tool-based options when questions span passages and structured relationships. Generated Cypher can fail, so plan for error handling.

These are fit-based recommendations, not claims that one approach will be more accurate for every dataset.

Product shortlist

1. LangGraph

LangGraph: A branching brass track fills a dim workshop as a cyberpunk cat pushes one glowing bead toward a pivoting gate. Several physical paths split and rejoin around the track, with the gate visibly directing the bead onward. The scene has crisp neon glows, plain unmarked surfaces, and no writing..LangGraph
LangGraph — AI editorial illustration, not a product photo or screenshot.

LangGraph suits agentic RAG when the next retrieval step depends on what the system finds along the way. Its graph structure lets you represent an agent as explicit steps connected by conditional routes, with shared state between them.

I like that this makes a changing retrieval workflow easier to inspect and customize than a single fixed search-and-answer chain. The trade-off is that you build the workflow yourself; LangGraph’s documentation does not promise that this design will make answers accurate.

What I like

  • Explicit nodes and conditional routing let you control what happens after each step.
  • Shared state supports passing information between stages such as search and response drafting.
  • Documented retry and tool-error loops provide building blocks for handling workflow failures.

Limitations

  • You must implement the application’s retrieval, state, and routing logic.
  • Retries and evaluation loops need to be designed by the developer, and do not guarantee answer accuracy.

My take

I’d consider LangGraph when retrieval needs to react to intermediate results, or when the workflow calls for explicit review or recovery steps. The documented example separates classification, document search, response drafting, and human review, illustrating how these decisions can be represented in a graph.

Best for

Developers building custom, stateful RAG workflows that need conditional retrieval steps and explicit control over what happens next.

Sources: Thinking in LangGraph.

2. Haystack

Haystack: A compact looping conveyor belt curves around a dark workshop bench as a cyberpunk cat guides one metal ball toward a padded catch pocket. The ball’s path visibly circles back before reaching the pocket, and the cat’s paw holds it at the stopping point. All parts are unmarked, with no writing..Haystack
Haystack — AI editorial illustration, not a product photo or screenshot.

Haystack suits teams that want a reactive RAG agent: the model can choose retrieval tools, use their results in later steps, and continue until a configured exit condition is met. I like this pattern when a query may need more than one retrieval action, but the system should still have a defined stopping point.

The framework provides controls for shaping that loop; it does not, by itself, ensure that the final answer is accurate.

What I like

  • The agent can iteratively call tools and update runtime state, which supports retrieval workflows that need to build on earlier results.
  • Configurable exit conditions and a maximum-step setting give developers a way to bound the loop rather than letting it run indefinitely.
  • Tool failures can either raise an exception or be passed back to the model, allowing developers to choose how errors are handled.

Limitations

  • Behavior depends on the configured model, tools, and exit conditions, so the loop still needs careful design and evaluation.
  • Passing tool errors back to the model may allow recovery attempts, but it is not a factual correctness check.

My take

The practical appeal is control over a straightforward tool-calling cycle: the agent can retrieve, inspect the result, and decide whether another step is needed. That makes Haystack worth considering when a fixed, one-shot retrieval pipeline is too rigid.

Best for

Choose Haystack if you want iterative retrieval-tool selection with explicit stopping controls. It is less suitable as a stand-alone answer to accuracy problems; you still need to evaluate the model, retrieval tools, and resulting answers.

Sources: Agent.

3. LlamaIndex

LlamaIndex: Three distinct woven baskets sit in a fan shape on a shadowy worktable as a cyberpunk cat lifts one small, textured cloth swatch from a matching basket. The other baskets hold different plainly visible materials, making the cat’s single choice clear. Neon light edges the scene; nothing bears writing or labels..LlamaIndex
LlamaIndex — AI editorial illustration, not a product photo or screenshot.

LlamaIndex is a retrieval-first framework that doesn't just answer your questions — it routes them. The RouterRetriever inspects the incoming query, picks one or more retrievers from your configured candidates, and executes them.

That makes it especially useful when different queries need specialized data sources or retrieval strategies rather than the same retrieval model run over every request.

What I like

  • Built-in adaptive routing across multiple candidate retrievers so some queries can hit vector, keyword, or hybrid retrieval based on their shape.
  • Lowers the cost of serving heterogeneous data by selecting retrievers per query instead of running all of them on every call.
  • Supports reading from multiple data sources in one pipeline without switching pipelines manually.

Limitations

  • The router selects only among the retrievers you supply; it does not independently evaluate generated answers for factual correctness.
  • Routing metadata rules are defined by you, which means incorrect routing patterns can still send queries to the wrong retriever.
  • For automatic query reformulation or graph reasoning within a single pipeline, LlamaIndex provides additional components but they are not built into the router itself.

My take

LlamaIndex's router is best when you have clearly distinct query types coming at your application: product lookup might need keyword search, while conversational follow-ups may benefit from vector retrieval. In those cases, routing materially outperforms a single generic retriever because each query reaches its most appropriate strategy.

I would pay particular attention to how explicit your routing metadata rules are, since misclassified queries still produce lower-accuracy answers. If your workload is simpler — mostly uniform Q&A — then a single retriever or even an external orchestration tool such as LangGraph may be enough, and adding routing overhead may not justify the complexity it introduces.

Best for

Developers who already use LlamaIndex or want to migrate toward one framework and need native routing across multiple data sources, vectors, keywords, or hybrid strategies.

Sources: Router.

4. Microsoft GraphRAG

Microsoft GraphRAG: A broad, unmarked jigsaw landscape covers a dark studio floor as a cyberpunk cat presses one final piece into the larger picture. The assembled puzzle shows connected hills, rivers, and paths across its surface, with no words or symbols. A vivid neon glow outlines the cat and puzzle pieces..Microsoft GraphRAG
Microsoft GraphRAG — AI editorial illustration, not a product photo or screenshot.

Microsoft GraphRAG’s global-search pattern is aimed at questions that require a view across a corpus, such as finding themes, rather than simply retrieving passages similar to a query. It builds on community reports in a knowledge-graph hierarchy and combines them through a map-reduce process.

I’d consider it when corpus-wide synthesis matters more than keeping the retrieval path simple.

What I like

  • The pattern is designed for aggregation across a dataset, which suits questions that may not be answered by a few locally relevant passages.
  • Community reports provide structured context for the global search process.
  • The documented map-reduce flow offers a clear approach to combining information from those reports.

Limitations

  • Global search is resource-intensive, and more detailed community levels can increase processing time and LLM resource use.
  • Answer detail can vary with the selected hierarchy level, so GraphRAG indexing does not guarantee correctness.

My take

I’d choose this pattern when users need themes or other high-level synthesis across a large corpus. For targeted questions where relevant passages are enough, a simpler retrieval approach may be a better fit.

Best for

Teams building corpus-wide research or analysis workflows that need to synthesize information across a dataset. Account for the indexing requirement and the extra resource use of global search.

Sources: Global Search.

5. Neo4j GraphRAG

Neo4j GraphRAG: A pegboard of interlinked brass rings and small metal objects hangs in a dim workshop as a cyberpunk cat pulls one key free from the connected chain. The neighboring rings visibly link the key to several different objects, while the cat makes one clear retrieval. Every surface is plain and unmarked, with no writing..Neo4j GraphRAG
Neo4j GraphRAG — AI editorial illustration, not a product photo or screenshot.

Neo4j GraphRAG is a good fit when retrieval needs to move between semantic document search and structured data, rather than relying on a single search method. Its Python package documents vector, hybrid, Cypher, and tool-based retrievers; a ToolsRetriever can use an LLM to select one or more tools, or none, and combine results with attribution.

I’d consider it for questions that span passages and relationships, while planning for query failures.

What I like

  • A choice of vector, hybrid, Cypher, and tool-based retrieval makes it possible to match retrieval to different kinds of questions.
  • ToolsRetriever can select among retrieval tools—or select none—and return attributed results, which helps make the retrieval path more visible.
  • Graph queries can complement semantic document search when answers depend on structured entities or relationships.

Limitations

  • Setup requires a Neo4j driver and configured retriever and LLM components.
  • Text2Cypher may generate syntactically invalid queries that fail at execution, so the application needs error handling. Attributed results do not by themselves guarantee a correct answer.

My take

The main advantage is the option to combine graph and vector retrieval instead of treating every question as a document-search problem. That can suit systems where users ask about both the meaning of passages and connections stored in a graph.

I would add failure handling for generated Cypher and evaluate whether tool selection retrieves the right evidence for your questions. The documented capabilities make this a flexible architecture, but they are not a substitute for checking answer quality.

Best for

Teams building RAG applications over data that mixes semantic passages with structured entities, relationships, or database queries, and that can support a Neo4j-based setup.

Sources: User Guide: RAG.

How to choose

How to Choose the Right Agentic RAG Pattern for Your Data

Sources: Thinking in LangGraph; Agent; Router; Global Search; User Guide: RAG.

Which option fits?

If your retrieval path needs to react to intermediate results, start by considering LangGraph for explicit workflow control or Haystack for a tool-calling loop with configured stopping rules. If different query types need different retrievers, LlamaIndex’s router is a more direct fit; if users ask questions about themes across a corpus, Microsoft GraphRAG’s global-search pattern is designed for that kind of synthesis.

For data that combines passages with structured entities or relationships, Neo4j GraphRAG offers a mix of retrieval approaches, but generated Cypher needs failure handling. Before choosing, map a representative set of queries to the evidence each should retrieve, then evaluate retrieval and answer quality with your own data.

None of these documented patterns makes that validation optional, and the more elaborate architecture is not automatically the better one.