Beyond Simple RAG: How Agentic RAG Answers Mongolian Law Questions

Simple retrieval-augmented generation works like a single library request: search once, hand the top passages to a language model, and ask it to answer. That is a strong starting point. It is also where many production systems discover the limits of one-shot retrieval.

A legal question rarely maps cleanly to one passage. The answer may require identifying the correct law, finding the relevant article, following an internal reference, comparing several provisions, and checking how courts applied them. If the first retrieval misses one link in that chain, the model cannot repair the evidence set.

Agentic RAG turns retrieval from a single event into an evidence-gathering loop. The model can search, inspect, read, reassess, and search again before it writes the answer.

This article explains the architecture we use for Mongolian-law research at huuli.tech, and the engineering decisions that make the loop useful rather than merely more expensive.

Simple RAG and agentic RAG

In simple RAG, application code controls retrieval:

Embed the question.
Retrieve the top matching chunks.
Put all retrieved text into the prompt.
Generate one answer.

In agentic RAG, the model controls the next retrieval action:

Decide what evidence is missing.
Select a typed tool.
Inspect the tool result.
Decide whether to retrieve more or answer.

The difference is not whether embeddings are used. An agent can use keyword search, database navigation, vector search, court search, and web search in the same investigation. The difference is who chooses the next piece of evidence and when the retrieval process stops.

	Simple RAG	Agentic RAG
Retrieval plan	Fixed in application code	Chosen step by step by the model
Context	One top-k bundle	Grows only with evidence the agent opens
Exact article lookup	Depends on ranking	Direct navigation and targeted reads
Multi-hop questions	Fragile	Can follow references across tools
Latency and cost	Lower and predictable	Higher and variable, but bounded
Best fit	Focused FAQ-style questions	Research, legal analysis, and complex documents

The production query loop

User question
    │
    ▼
Legal agent receives conversation + tool schemas
    │
    ▼
What evidence is missing?
    │
    ├── find law by name ──► inspect law map ──► read exact article(s)
    │
    ├── keyword search ─────► open candidate article(s)
    │
    ├── semantic search ────► reranked law passages + cited clauses
    │
    ├── court search ───────► cassation ──thin?──► appellate
    │
    └── prior grounding ────► reuse evidence from an earlier turn
    │
    ▼
Enough verified evidence?
    ├── No: choose another tool and continue
    └── Yes: synthesize answer with clickable citations

This is a loop, not an unrestricted autonomous process. Each tool has a name, a description, a validated input schema, and a server-controlled implementation. The model can choose among those capabilities, but it cannot invent a new database operation or bypass authorization.

At each step the language model emits either a native tool call or prose. The application executes the tool, appends its structured result to the conversation, and runs the model again. A hard step budget prevents runaway research. The first step of a substantive legal question is forced to use a tool, while the final allowed step disables tools and forces the model to answer from the evidence already collected.

That last guard matters. Without it, an agent can consume its entire budget requesting one more search and terminate without producing a useful answer.

The Mongolian-law toolset

The strongest design choice was to avoid making every tool a vector search. Legal corpora already have structure: law names, chapters, articles, clauses, citations, courts, and instance levels. The tools should expose that structure.

1. Find a law by name

When a user mentions the Criminal Code or Labour Law, the agent first resolves the authoritative internal law identifier. It is explicitly forbidden from guessing one.

The result is a small candidate list with titles, summaries, and article counts. This is cheap, deterministic navigation.

2. Inspect the law map

A code may contain hundreds of articles. Sending the entire law to the model wastes tokens and makes relevant provisions harder to see.

The law-map tool returns only:

the law title and short summary;
status and article count;
article numbers and headings.

The agent uses this table of contents to decide which provisions to open.

3. Read exact articles

Once the agent knows the relevant article numbers, it reads one article or a batch of articles. These tools return the complete provision and its authoritative Legalinfo link.

This creates an important evidence boundary: the final answer may cite the exact text the agent actually opened, rather than a number remembered by the language model.

4. Search by keyword

Keyword search is often better than embeddings for exact legal terminology. Mongolian is agglutinative, so the search uses prefix matching to catch inflected forms while filtering common legal boilerplate.

The result contains anchors and short snippets, not full article bodies. The agent must deliberately open the relevant article before relying on it.

5. Search by meaning

Semantic search remains valuable for abstract questions such as the general requirements for forming a contract. This tool runs a conventional RAG pipeline:

Embed the query.
Over-fetch candidate law passages with pgvector.
Rerank candidates against the original question.
Expand internal statutory references.
Return full, citable evidence.

Agentic RAG therefore does not replace simple RAG. It makes RAG one instrument inside a broader research process.

6. Search court decisions

Statutory text says what the rule is. Court decisions show how the rule is applied.

The court tool accepts a Mongolian query plus optional court family and instance level. By default it searches cassation decisions first. If fewer than three relevant decisions are found, it expands to appellate decisions. First-instance decisions are available only when explicitly requested because they carry less authority and may have been overturned.

The query is embedded once and reused across the escalation ladder. If vector retrieval finds nothing, full-text search runs once at the end. The tool reports which levels it searched and where the results came from, allowing the answer to distinguish a high-authority cassation ruling from a lower-court example.

7. Reuse prior grounding

Follow-up questions should not repeat expensive searches merely because the user says, “Summarize those decisions.”

Each completed turn persists the sources actually read. Later turns receive a compact manifest of prior searches and can retrieve one by number. The agent can therefore reuse the exact earlier evidence, including its citation tags, instead of reconstructing or re-searching it.

8. Use specialized sub-agents where the corpus differs

Tax law includes statutes, government resolutions, ministerial rules, tax-authority orders, appendices, rates, and exemptions. It has its own navigation behavior, so the main agent delegates tax questions to a specialized inner agent.

The same pattern works for separate jurisdictions: keep the outer tool list flat, expose one high-level research tool per jurisdiction, and let the inner agent navigate that corpus using its own citation rules.

Why progressive disclosure beats dumping context

The core pattern is:

search → inspect map → read exact source → answer

Search tools return candidates. Map tools return structure. Read tools return authoritative text. This separation keeps early steps small and makes the model spend tokens only on provisions likely to matter.

It also makes failures visible. If search returns no candidate, the agent can say so or try a different search strategy. If an article cannot be opened, the agent cannot quietly cite it anyway. In a large prompt containing dozens of chunks, those boundaries are much harder to enforce.

Two embedding spaces, one research experience

The law and court corpora were embedded at different times and use different vector spaces:

Corpus	Model	Dimensions	Retrieval framing
Mongolian law chunks	Gemini embedding 001	1,536	Semantic similarity
Court-ruling summaries	Gemini embedding 2	512	Asymmetric search query/document prefixes

These vectors cannot be mixed. A query must be embedded with the exact model, dimension, task framing, and distance metric used for its target corpus.

For court search, stored ruling summaries use a document prefix, while user queries use a different search-query prefix. That asymmetry is intentional: the query asks for a result, while the document represents a candidate result. Using the wrong prefix can return plausible but poor matches without producing an obvious error.

The agent hides this infrastructure detail from the user. It chooses “search statutes” or “search courts”; the tool implementation selects the correct embedding contract.

Grounding is a system property, not a prompt request

“Please do not hallucinate” is not an adequate reliability mechanism.

The system adds structural controls:

substantive legal turns must retrieve before answering;
law identifiers must be discovered, never guessed;
provisions can be cited only with links returned by tools;
court decisions carry self-contained citation tags;
lower-court decisions must be described at their actual authority level;
articles and rulings actually read are recorded separately from text merely considered;
an empty search must degrade to an honest “not found,” not a fabricated citation;
a maximum step count bounds cost and latency;
the final step is reserved for synthesis.

The model still exercises judgment: which law is relevant, which article to open, whether court practice is needed, and whether the evidence answers the question. But the application owns identity, authorization, retrieval, citation construction, persistence, and budgets.

When agentic RAG is worth it

Use simple RAG when one retrieval pass normally contains the answer: support documentation, a narrow product manual, or a stable FAQ corpus. It is faster, cheaper, and easier to evaluate.

Use agentic RAG when questions regularly require:

exact source identification before retrieval;
navigation through long structured documents;
multiple provisions or cross-references;
statute plus precedent;
different retrieval methods for different evidence types;
follow-up research based on earlier findings;
an honest recovery path when the first search fails.

The practical rule is simple: add agency only where another retrieval decision can materially improve the evidence. More steps are not automatically better. A useful legal agent stops as soon as it has enough authoritative material to answer.

The larger lesson

The value of agentic RAG is not that the model “thinks harder.” Its value is that the system gives the model a controlled way to obtain better evidence.

For Mongolian law, that means combining the strengths of several retrieval modes:

deterministic navigation for known laws and articles;
keyword retrieval for exact terminology;
semantic retrieval for concepts;
court search for real-world application;
persistent grounding for follow-up questions;
strict citation and authority rules for the final answer.

The result behaves less like a chatbot with a large context window and more like a careful research workflow: locate, inspect, read, verify, and only then explain.

That is the shift from RAG to agentic RAG.