Reliability for retrieval & grounding: recall, precision, citations
This hub focuses on reliability of retrieval and grounding: recall/precision, chunking, hybrid retrieval, reranking, source attribution, and eval for RAG. For teams using RAG who can't trust it yet.
Common "can't trust RAG" symptoms
Low recall / low precision (and how to fix them)
When RAG retrieval fails, the answer is wrong downstream. Diagnose the dominant failure mode first.
Low recall
Relevant docs exist but never surface. Query-doc mismatch, embedding gap, or corpus coverage holes.
Fix: Improve query expansion, hybrid search (BM25 + vector), chunking strategy, or embedding model.
Low precision
Top-k is noisy; irrelevant chunks crowd out good ones. Model gets confused by weak context.
Fix: Reranking, stricter retrieval filters, better chunk boundaries, or smaller top-k with higher quality.
Wrong doc rank
Right doc retrieved but ranked too low. Reranker or scoring function mis-calibrated.
Fix: Tune reranker, adjust score fusion (RRF, weighted), or add relevance feedback.
Stale / missing corpus
Knowledge base outdated or missing key sources. New docs not indexed; deletions not reflected.
Fix: Freshness pipeline, incremental indexing, coverage audits, and change detection.
Query drift
User query rewritten or expanded in a way that loses intent. Query understanding fails.
Fix: Query rewrite rules, intent-aware retrieval, or A/B test retrieval with/without expansion.
Embedding drift
Embedding model or corpus changes; semantic space shifts. Old evals no longer match production.
Fix: Monitor embedding drift, periodic re-eval, and versioned embedding pipelines.
How you chunk and index shapes retrieval quality
Chunk size, overlap, and boundaries affect retrieval precision and recall. Choose the right pattern for your corpus.
Fixed-size chunks
Simple token-based splitting. Fast but can break semantic units. Overlap helps.
Semantic / sentence boundaries
Split on sentences or paragraphs. Better coherence; variable chunk sizes.
Document-aware chunking
Respect document structure (headers, sections). Preserves hierarchy.
Parent-child / hierarchical
Small chunks for retrieval; parent chunks for context. Best of both.
Trade-offs: recall vs precision vs latency
BM25 + vector fusion improves recall. Reranking improves precision. Best reranker for RAG latency tradeoff: smaller top-k → faster; larger top-k → more recall.
BM25 + vector fusion
Hybrid search combines lexical (BM25) and semantic (vector). RRF or weighted fusion.
Reranking models
Cross-encoder or LLM reranker for top-k. Higher precision but adds latency.
Two-stage retrieval
Stage 1: broad recall (e.g. vector top-50). Stage 2: rerank to top-5.
Source attribution and context grounding
Citations for RAG must be accurate and verifiable. Reduce RAG hallucinations by enforcing grounding and attribution.
Source attribution
Map each claim to specific retrieved chunks. Citation-to-claim verification.
Groundedness checks
Detect when answers go beyond context. Ungrounded rate as a core metric.
Citation format
Structured output: [1], [2] or inline links. Parsing and validation downstream.
Refusal when uncertain
Model should abstain when context is insufficient. Reduces confident wrong answers.
Golden set from logs (and how to build it)
RAG evaluation requires a representative test set. Build it from production logs, failures, and known-good examples.
What to include in your RAG eval dataset
- 1Golden set from production logs (sampled failures + successes)
- 2Synthetic queries from FAQ, support tickets, or user sessions
- 3Human-labeled relevance (query → doc pairs) for retrieval eval
- 4End-to-end task success (can the model answer correctly with this context?)
- 5Regression suite: known-good and known-bad examples
RAG quality degrades over time — catch it early
Embedding drift, corpus drift, and query distribution drift. Monitor and maintain with a cadence.
Embedding drift
Embedding model or corpus changes; similarity distribution shifts. Monitor embedding consistency over time.
Corpus drift
New docs added, old removed. Coverage and freshness need periodic audits.
Query distribution drift
User queries change over time. Eval set should reflect production traffic.
Maintenance cadence
Re-eval weekly or monthly; re-index on schema or model changes; version embeddings.
Maintenance cadence
Re-eval weekly or monthly; re-index on schema or model changes; version embeddings. Embedding drift is silent until you measure it.
Practical guides
Stop guessing. Use these diagnostics to classify retrieval failures, choose a chunking strategy, and fix the right thing.
Proof before the CTA
RAG pillars convert better when they show a real fix path
These are the proof assets this hub should route to: retrieval recall recovery and grounded-answer improvements with concrete artifacts, not just theory.
Need a RAG audit or retrieval improvement?
If your RAG retrieval is missing key docs, grounding is weak, or citations are wrong, we can diagnose and fix retrieval quality, chunking, and evaluation.
Start → Fix → Govern
Enforce the Audit → Sprint → Retainer ladder
Enterprise outcomes require a baseline, shipped fixes, then governance. This is the shortest path to measurable quality, controlled cost, and regression prevention.