The core idea
Recall and precision are two competing forces. Diagnose which one you have before you fix—increasing k often backfires.
When a RAG system starts giving wrong answers, most teams do the same thing: increase k, add reranking everywhere, add more documents, tweak prompts. Sometimes it helps. Often it makes things more expensive, slower, and still wrong.
The reason is simple: you're treating retrieval quality as one thing, when it's actually two competing forces. This post gives you a practical RAG debugging guide to determine recall vs precision vs context construction—and what to fix first.
What is Recall in RAG?
Recall measures whether your retrieval system successfully surfaces the document that contains the correct answer. High recall means the right source appears in your candidate set.
What is Precision in RAG?
Precision measures how much of the retrieved context is actually relevant, and whether irrelevant chunks confuse the model. High precision means most chunks support the answer; low precision means junk dilutes or contradicts the right information.
In production systems we audited (finance, SaaS, internal knowledge assistants), most RAG failures were not embedding problems—they were selection and context construction failures. Diagnosing recall vs precision first saves weeks of wrong fixes.
Context
Part of the RAG Reliability hub: RAG Reliability Hub. See also: Why RAG Still Hallucinates When Retrieval Looks Fine, RAG Optimization Service (fix order), RAG Wrong Answers Triage (debugging guide), Audit Readiness, AI System Audit Deliverables, and our production AI reliability services.
RAG evaluation metrics (the 5 you need)
To diagnose RAG reliability, track these five metrics:
- Candidate Recall @50 — Is the right source in the top 50 retrieved candidates?
- Selection Recall — Was the right source included in the final context passed to the model?
- Context Precision Ratio — What % of chunks in the final context are relevant (vs junk)?
- Contradiction Rate — How often does retrieved context contain chunks that conflict with the correct answer?
- Coverage Rate — What % of queries have a "right source" in your corpus at all?
These metrics map directly to the diagnostic steps below. For deeper theory, see Stanford IR and retrieval evaluation in the LLM era.
The two failures that look identical (but aren't)
Recall failure: "We missed the answer"
The correct document exists, but retrieval never surfaces it in the candidate set.
Typical symptoms: the system answers confidently but cites irrelevant content; users say "it's in our docs" but the assistant doesn't find it; increasing k sometimes helps (but can blow up cost).
Precision failure: "We retrieved too much junk"
Retrieval returns documents that are loosely related but not sufficient to answer.
Typical symptoms: the system includes lots of context but still answers wrong; answers are inconsistent (the model latches onto different chunks); increasing k makes answers worse.
Most systems have both issues—just in different cohorts.
Example: Pricing Policy Failure
- Correct doc exists in the corpus
- In top 20 candidates (retrieval fine)
- Not selected due to reranker diversity constraint
- Model answered using an outdated blog post instead
→ Selection recall failure, not recall or precision. Fix: adjust selection logic, not embeddings.
The 30-minute diagnostic (no guessing)
Run this on 20–50 real production queries that were: reported as wrong, or labeled low-confidence, or escalated to a human. If you don't have those yet, sample: 10 "common" queries, 10 "hard" long-tail queries, 10 "high-risk" queries (policy, pricing, compliance).
What you need to log (minimum)
For each query: candidate retrieval results (top N doc/chunk IDs + scores), final selected context (what actually goes into the prompt), answer + citations (if any). If you don't have this logged yet, implement the minimum schema first—see Audit Readiness.
Step 1 — Define what "the right source" means (fast)
For each query, decide whether there exists a "right source" in your corpus: Right source exists (a specific doc/section has the answer) or Right source does not exist (your KB lacks it—coverage problem).
Output: Coverage rate: % queries where the answer exists in KB. If coverage is low, you don't have a retrieval problem—you have a content coverage problem.
Step 2 — The "candidate recall" check (top N)
For each query where the answer exists in the KB: Is the right source present anywhere in the top N candidates? Pick N = 50 for the diagnostic (even if your production k is smaller). This is your candidate recall.
Result: If the right source is not in top 50 → Recall problem. If it is in top 50 → move to Step 3.
Step 3 — The "selection recall" check (what enters the prompt)
Was the right source included in the final context passed to the model? This identifies ranking/selection failure: reranker issues, selection heuristics issues, context budget truncation.
Result: In top 50 candidates but not selected → ranking/selection issue. Selected but model still wrong → context construction or generation issue.
Step 4 — Precision check (junk ratio)
Precision isn't "did we get one relevant chunk." It's: how much irrelevant context did we include? For each query: Relevant chunks (directly support the answer), Irrelevant chunks (don't support or distract), Contradictory chunks (conflict with the right source—worst case).
Quick rubric: High precision: 70%+ chunks relevant, no contradictions. Medium: 40–70% relevant, some noise. Low: <40% relevant or contradictions present. If precision drops hard when you increase k, you're confusing the model.
Diagnostic flow
Query ↓ Answer exists in KB? ↓ Yes In top 50 candidates? ↓ Yes Selected into final context? ↓ Yes Precision high (70%+ relevant)?
Each "No" points to a different fix: coverage → recall → selection → precision.
def candidate_recall(query, top_k=50):
candidates = retrieve(query, k=top_k)
return contains_right_source(candidates)The diagnostic matrix (how to interpret results)
Pattern A: Low candidate recall (top 50 misses)
Meaning: Embeddings/search can't find the right source reliably. Causes: chunking mismatch, embedding model mismatch, missing metadata filters, no hybrid search, stale index. Fix order: content hygiene + ingestion, chunking strategy per doc type, hybrid search or query expansion, embedding model selection.
Pattern B: Candidate recall fine, selection recall low
Meaning: You retrieved the right thing, but it didn't reach the model. Causes: reranker not calibrated, selection logic picks diverse but shallow chunks, context budget truncation, dedupe issues. Fix order: selection rules + context budget allocation, reranker routing, "must-include" logic, better dedupe and ordering.
Pattern C: Selection recall fine, precision low
Meaning: Context contains too much junk. Causes: k too high, reranking doesn't reduce noise, multi-hop without constraints, duplicate chunks, no confidence gating. Fix order: cap context tokens + chunk count, confidence gating, dedupe + contradiction checks, reranking targeted to low-confidence cohorts.
Pattern D: Retrieval is fine, answers still wrong
Meaning: Generation/prompt/validation issue (not retrieval). Causes: model overreach, prompt hierarchy conflicts, lack of citation enforcement, output format not validated, tool calling introduces wrong state. Fix order: citation verification, output validation, prompt hierarchy cleanup, model routing for high-risk intents.
Recall vs precision failure matrix (2×2)
| High Precision | Low Precision | |
|---|---|---|
| High Recall | Good system | Noise problem — cap k, improve reranking |
| Low Recall | Retrieval failure — chunking, hybrid search, embeddings | Broken system — fix recall first |
Problem summary table
| Problem Type | Root Cause | First Fix | Cost Impact |
|---|---|---|---|
| Low candidate recall | Chunking, embeddings, hybrid search | Content hygiene, chunking strategy | Higher k → 30–50% token cost increase |
| Low selection recall | Reranker, context budget, dedupe | Must-include logic, selection rules | Wrong answers → support/compliance risk |
| Low precision | k too high, noise, contradictions | Cap context, confidence gating | 10% precision drop → 30–50% token cost |
| Coverage gap | KB missing content | Content audit, ingestion | 5% recall gap in policy → compliance risk |
In high-volume systems, a 10% precision drop can increase token cost by 30–50%. A 5% recall gap in policy queries can become a compliance risk. Enterprise teams should treat these as cost and risk drivers, not just technical metrics.
Why "just increase k" often backfires
Increasing k improves candidate recall but can crush precision: more irrelevant chunks, more contradictions, more tokens (higher cost), longer context (higher latency), model becomes less deterministic.
Better approach: Adaptive retrieval—start small, retrieve more only when confidence is low, budget context per intent/risk tier.
Not sure whether you have a recall or precision problem?
Start with the 50-query diagnostic template below. Or get the RAG Reliability Diagnostic Sheet (Google Sheet template) to run this systematically.
Practical fixes you can ship fast (without replatforming)
1) Add two retrieval modes (fast vs deep)
Fast mode (default): low k, no rerank, strict metadata filters, capped context tokens. Deep mode (only when needed): higher k, rerank on, query rewrite or expansion, multi-hop only for specific intents. Decide mode by: low retrieval confidence, high-risk intent, user asks for citations, previous attempt failed.
2) Context budgets (the simplest cost + quality win)
Define: max chunks = 6–10, max RAG tokens = 1,200–2,500, reserve tokens for system + user + tool traces. Enforce: dedupe, must-include top chunk, drop contradictions if doc versioning available.
3) "Stop retrieving when confident"
If top chunks strongly match and support the answer, don't keep retrieving. Signals: high score gap between top results and the rest, consistent doc type + recency, citation check passes on first attempt.
The output of a good diagnostic (what you should end with)
- Coverage rate (do we even have the answers?)
- Candidate recall (top 50 contains the right source?)
- Selection recall (did the right source reach the model?)
- Precision score (junk ratio / contradictions)
- A ranked fix list: 3–5 changes with the biggest ROI
- Validation plan: how you'll prove improvements (eval set + cohorts)
Who this diagnostic is for
This diagnostic is for:
- AI leads deploying internal copilots
- SaaS teams shipping customer-facing RAG
- Enterprises under compliance constraints
- Teams spending >$5k/month on LLM infra
A quick worksheet (copy/paste)
Use this table for 20–50 queries:
Query: ____
Answer exists in KB? (Y/N)
Right source in top 50 candidates? (Y/N)
Right source selected into final context? (Y/N)
Precision score (High/Med/Low): ____
Contradictions present? (Y/N)
Likely issue: Coverage / Recall / Ranking / Precision / Generation
Suggested fix: ____
Want us to run this diagnostic on your production traces?
Soft: Not sure where to start? Download the RAG Reliability Diagnostic Sheet (50-query template) and run it yourself.
Mid: We can run this on your traces in 5 business days—recall/precision breakdown by cohort, top failure modes, fix roadmap, and before/after validation.
Hard: Book a RAG Reliability Assessment Call for a full diagnostic on your production system.
If you share ~200 anonymized queries and retrieval logs (candidate IDs + selected chunks) for 1–2 days, we deliver: recall/precision breakdown by cohort, failure taxonomy, fix roadmap (chunking, hybrid search, reranking, budgets), and evaluation to prove impact.
Next step
Request a RAG Reliability Assessment or start with an AI System Audit.
FAQ
Questions readers usually ask next
What is recall in RAG?
Recall in RAG measures whether your retrieval system successfully surfaces the document that contains the correct answer. High recall means the right source appears in your candidate set; low recall means retrieval misses it entirely.
What is precision in RAG?
Precision in RAG measures how much of the retrieved context is actually relevant, and whether irrelevant chunks confuse the model. High precision means most chunks support the answer; low precision means too much junk dilutes or contradicts the right information.
What's the difference between recall failure and precision failure in RAG?
Recall failure: the correct document exists but retrieval never surfaces it in the candidate set. Precision failure: retrieval returns loosely related documents that are not sufficient—too much junk confuses the model. Increasing k sometimes helps recall but often hurts precision.
What is candidate recall vs selection recall?
Candidate recall @50: is the right source in the top 50 candidates? Selection recall: was the right source included in the final context passed to the model? If it's in top 50 but not selected, it's a ranking/selection issue.
Why does increasing k make RAG answers worse?
Higher k improves candidate recall but can crush precision: more irrelevant chunks, more contradictions, more tokens (cost + latency), and the model becomes less deterministic. Use adaptive retrieval instead—start small, retrieve more only when confidence is low.
Want us to run this diagnostic?
Share ~200 anonymized queries and retrieval logs. We deliver recall/precision breakdown by cohort and a fix roadmap. See our AI System Audit service.
Last updated
February 17, 2026
