The core idea
Production AI rarely fails in one dramatic place. It leaks across quality, cost, latency, release safety, and governance. A checklist helps you see the pattern early.
Most production AI systems do not fail loudly. They fail quietly: a few wrong answers in the right customer account, a few extra retries per request, a little more context stuffed into every call, a little less trust after every release.
That is why a production LLM audit starts with signals, not opinions. If you cannot tell whether the system is leaking money, trust, or both, the team will default to random prompt edits and vendor blame. This checklist is designed to stop that pattern.
Context
This article is part of the LLM Audit hub. Related reading: Do You Need an LLM Audit?, OpenAI Bill Audit, RAG Wrong Answers Triage.
How to use this checklist
Read each signal and mark it as one of three states:
- No: we have this under control and can prove it
- Partial: we see the issue sometimes or only for some cohorts
- Yes: this is currently true in production
The rule is simple: the more signals you mark Partial or Yes, the less you should trust anecdotal success. Production AI systems usually break at the edges first: long-tail queries, hard cohorts, overloaded windows, prompt changes, or security review.
Quality and trust signals
1) Users say answers are wrong, but internal demos still look fine
This usually means you are measuring averages instead of cohorts. The system looks fine in happy-path demos and quietly fails on real customer queries, long documents, weaker retrieval, or messy tool outputs.
2) The system sounds confident when it should refuse
Wrong answers are bad. Confidently wrong answers destroy trust faster. If refusal logic, citation gating, or answer confidence rules are weak, the product will over-answer in situations where grounded evidence is missing.
3) Citations exist, but they do not actually support the claim
Teams often mistake "has citation" for "is grounded." A source link is not proof. If the cited span does not support the answer, your trust layer is cosmetic.
4) One prompt fix improves one workflow and breaks another
This is a classic sign of missing eval segmentation. You are treating the system like a single task when it is really multiple task families with different failure modes.
5) Support or customer-success teams do not trust the AI enough to rely on it
If internal teams verify everything manually, the feature may look "adopted" in dashboards while delivering little actual leverage. Trust debt is still debt.
Cost and unit-economics signals
6) Your AI bill is rising faster than usage
Cost should roughly track value-producing volume. If spend rises faster than sessions, tickets deflected, or tasks completed, something is bloating the path: retries, oversized prompts, wasted reranking, weak routing, or bad caching.
7) Nobody can explain cost per successful task
Cost per request is too shallow. You need cost per successful outcome, or the team will optimize cheap failures and miss expensive low-quality flows.
8) The largest model is still the default for easy work
If simple intents always hit the most expensive model, you probably have no routing discipline. That is rarely a quality decision; it is usually an operations shortcut.
9) Retrieval parameters keep increasing, but answer quality barely moves
Increasing k, adding more chunks, or reranking more candidates without measurement is a common way to buy latency and token cost without solving the real recall or context problem.
10) Retries and tool loops are invisible in dashboards
Hidden retries are one of the fastest ways to leak cost. If they are not broken out by stage, the bill looks random and nobody can see the feedback loops.
Latency and serving signals
11) P50 is acceptable, but P95 or P99 is painful
Tail latency is where production UX breaks first. This often points to queueing, slow tools, rerank bottlenecks, cold starts, or retry storms rather than just "the model is slow."
12) Time-to-first-token and full response time are not tracked separately
If TTFT and full completion are blended into one number, you cannot tell whether the user is waiting on retrieval, model start-up, or long generation paths.
13) Latency gets worse during specific intents, tenants, or document types
That usually means you have cohort-specific pipeline complexity. The bottleneck may be tied to document size, tool selection, or a retrieval path that only some customers trigger.
14) Streaming exists, but the user still perceives the product as slow
Superficial streaming can hide a weak first token, poor intermediate content, or long waits before useful substance appears. Streaming is not a substitute for stage-level latency work.
15) Timeout and fallback policies are unclear or inconsistent
If some flows retry forever, some hard-fail, and some silently degrade, the product will feel unpredictable under load. Reliability requires explicit latency budgets and fallback behavior.
Observability and diagnosis signals
16) You cannot classify failures as retrieval, generation, or tool failures quickly
If every issue turns into prompt tuning, you are operating without a diagnostic model. A real audit needs failure-layer visibility.
17) There is no saved trace that reproduces a bad answer end-to-end
Without request-level traces, your postmortems become storytelling. You need to see the query, selected chunks, tool calls, timing, model output, and validation outcome in one chain.
18) Metric dashboards stop at request count, latency, and cost
Those are platform metrics, not system understanding. You also need groundedness, task success, refusal quality, tool success rate, cohort labels, and regression indicators.
19) The team has no agreed failure taxonomy
If engineers, PMs, and support use different language for the same failures, prioritization drifts. A taxonomy creates leverage because repeated issues become measurable classes.
20) You only discover problems after customers report them
This means monitoring is behind the user. Production AI needs early-warning indicators: eval drift, latency burn, retry spikes, retrieval recall proxies, or tool error changes.
Release safety and control signals
21) Prompt or model changes ship without regression gates
Every un-gated release is an experiment on customers. If you lack a golden set, rubrics, and approval thresholds, you are shipping blind.
22) There is no versioned baseline for prompts, models, and retrieval settings
If the team cannot answer what changed between "working" and "broken," recovery slows down and blame rises. Versioning is basic operational hygiene.
23) Teams optimize quality, cost, and latency in separate conversations
This creates local wins and system losses. An audit forces the tradeoff into one scorecard so you can see whether a fix actually improved the product.
Security and governance signals
24) Security reviews stall because nobody can explain logging, PII, or vendor boundaries
This is a trust leak even if the system works. Enterprise buyers need clear answers on what is logged, redacted, retained, and sent to model vendors.
25) Prompt injection and unsafe tool execution are treated as edge cases
If retrieval can import hostile instructions or tools can act on unvalidated output, the system has governance debt. Production AI should assume adversarial input exists.
Scoring: when to escalate to a real audit
Count every Yes as 2 points and every Partial as 1 point.
- 0–4: The system may be healthy, but keep monitoring and validate edge cohorts.
- 5–9: You have meaningful blind spots. Instrumentation and tighter controls should start now.
- 10–17: You are likely leaking money or trust already. A formal audit should be prioritized.
- 18+: Stop tuning around symptoms. You need a production baseline, root-cause map, and a fix roadmap before scaling further.
One more rule: if any single signal affects executive reporting, enterprise security review, or a core customer workflow, do not wait for the total score to get high. Escalate early.
What a real audit should produce next
A checklist tells you whether you have a problem. An audit tells you where the problem lives, how expensive it is, and what to fix first.
- Baseline scorecard: quality, groundedness, cost per successful task, P95 latency, and cohort splits
- Failure taxonomy: top failure classes with real examples and estimated impact
- Trace-backed diagnosis: retrieval vs generation vs tools vs serving
- Fix roadmap: fast wins, structural fixes, and tradeoffs between quality, cost, and latency
- Control plan: eval gates, release checklist, observability upgrades, and security controls
Next step
If this checklist surfaced more than a few weak spots, use it as your intake document for a real audit. Bring examples of wrong answers, cost spikes, slow traces, and failed releases. That is enough to start a proper baseline.
FAQ
Questions readers usually ask next
When should we run this LLM audit checklist?
Run it before a budget review, before scaling usage, after a spike in wrong answers or latency, after a model or prompt migration, or any time leadership asks whether the system is actually creating value. It is especially useful when the team feels quality is unstable but cannot explain why.
How many checklist signals are too many?
If 0–2 signals are true, keep monitoring. If 3–5 are true, you likely have local weaknesses and should instrument them now. If 6–10 are true, you are almost certainly leaking money or trust in production. If more than 10 are true, stop guessing and run a formal audit with baseline metrics, failure taxonomy, and a fix roadmap.
What should a real LLM audit deliver after this checklist?
A real audit should produce a measurable baseline for quality, groundedness, cost per successful task, P95 latency, and cohort breakdowns; a failure taxonomy with examples; stage-level traces; a prioritized fix roadmap; regression gates; and a logging or security plan where needed.
What this checklist is for
Early diagnosis. Use it to decide whether you need deeper instrumentation, a targeted sprint, or a formal AI system audit.
What this checklist is not
It is not a replacement for baseline measurement. It tells you where risk likely exists, not how much each failure mode costs.
Need a trace-backed audit?
We baseline quality, cost, latency, and failure modes, then turn that into a prioritized roadmap with before/after proof. See the AI Production Audit and the sample deliverables.
Last updated
March 7, 2026
