LIVE DEMO · REAL DOCUMENT · REAL LLM

An answer you can verify, not one you have to trust.

rag-guard is a typed-decision layer for RAG. Before you see an answer, it checks that retrieval found something relevant, that there is enough of it, and that every claim is backed by the source.

7 bounded decisions 1 LLM call 100% logged and auditable
attention_is_all_you_need.pdf63 chunks
What BLEU score did the Transformer reach on English-to-German?
ANSWERED · 1/1 CLAIMS SUPPORTED

The Transformer reached a BLEU score of 28.4 on the WMT 2014 English-to-German translation task.

Intentp 1.00
Relevance8/14 kept
Sufficiencyp 0.97
Caveatp 0.48
THE PROBLEM

Most RAG hallucination isn't the model lying. It's an answer built on the wrong context.

Vector search finds passages that are close to a query in embedding space. That is not the same as passages that answer it, or an answer the passages actually support.

SILENT GAP

Retrieval misses the one chunk that answers the question, and nothing downstream knows to say so.

CONFIDENT GUESS

The context is too thin, but the model answers anyway: fluently, and wrong.

UNVERIFIABLE CLAIM

The answer reads fine, but you can't tell which parts the source text supports.

HOW IT DECIDES

Seven calibrated judgments. Four of them run before the LLM does.

Each stage returns a probability you can threshold and log, not prose you have to parse. A "no" at any gate stops the pipeline before a guess is generated.

GATES · BEFORE GENERATION
1
Intent
Should this query reach retrieval, or be escalated to a human or refused outright?
proceed / escalate / refuse
2
Relevance
Does each retrieved passage help answer the query? Drops noise early.
kept / dropped per chunk
3
Clarify
Is the query too ambiguous to answer, however good the context is?
clarify
4
Sufficiency
Is the kept context enough: fully, partially, or not at all? A "no" skips generation.
insufficient_context
GENERATION · ONE LLM CALL
5
Generate
Runs only against the chunks that survived relevance filtering.
1 LLM call
CHECKS · AFTER GENERATION
6
Grounding
Splits the answer into claims and checks each against the context. Flags drift.
ungrounded_answer_flagged
7
Caveat
Even a grounded answer may need a caveat: partial coverage, or an edge the source hedges on.
answered_with_caveat
TWO WAYS TO SEE IT WORK

Bring a document neither rag-guard nor the model has seen.

There is no pre-built demo corpus, on purpose. Upload a .txt, .md or .pdf.

Single provider

Chat normally. Every reply carries a visible reasoning trace.

Chunk 10.99
Chunk 330.95
Chunk 350.20
  • Which chunks were kept or dropped, and at what probability
  • The sufficiency gate against its threshold
  • A per-claim grounding breakdown

Compare mode

Run one question through two embedding providers at once.

Raw answerguard off
Raw answerguard off
GuardedOpenAI
GuardedGoogle
  • text-embedding-3-small vs gemini-embedding-001
  • Same chunks, thresholds and generator: only retrieval differs
A REAL DEPENDENCY

Installed over pip, pinned to a commit. Not embedded logic.

This app depends on rag-guard like any other project would, to prove the published package behaves as its docs and tests claim.

from rag_guard import RagGuard

guard  = RagGuard(model=decision_model)
report = guard.run(query, chunks, generate_fn)

assert report.action in (
    "answered", "answered_with_caveat", "insufficient_context",
    "ungrounded_answer_flagged", "clarify", "escalate", "refuse",
)

The judgment calls come from a pluggable decision model. Swap it without changing pipeline code:

Heuristic (no API key) OpenAI Anthropic TypeSafe's Jev (native typed decisions)

Ask your document something hard.

Then open the trace and see exactly why it answered, or why it refused.

Open the app