Skip to main content

​
Inspect grounding citations on an AI agent

When an AI agent answers out of your knowledge base, every turn records which knowledge chunks the model retrieved and which of those it actually cited in the reply. The grounding-citations surface is where you read that record — per turn, per agent, and across the whole tenant. Use it when a wrong answer needs a source-level explanation, or when you want to verify the agent is standing on the knowledge you attached and not improvising. This guide walks the dashboard surface (Agents → RAG Grounding Citations), the verdict model behind each row, and the API endpoints that back it. The per-turn verdicts concept is covered in depth in Per-turn grounding citations; this page is the operator walkthrough for the audit surface itself.

​
Open the page

In the dashboard, open Agents and pick RAG Grounding Citations in the agents group. The page aggregates every voice-agent turn in the selected window into four blocks:
  • Totals strip — voice turns, the share of turns where retrieval fired, citation coverage (turns whose reply carried at least one citation marker), and the no-retrieval share.
  • Recent ungrounded turns — the newest turns where retrieval fired but the reply carried no citation marker, cross-referenced with retrieval confidence when the post-hoc scoring pass ran. This is the triage list.
  • Daily trend — voice-turn volume per day with the grounded share filled in, so a coverage dip lines up with the day it started.
  • By agent — citation coverage and retrieval share per voice agent, so a single misbehaving agent is not hidden inside a healthy fleet average.
Access is owner or admin only, because the rollup spans the whole tenant’s grounding behaviour. A member role sees a permission message rather than an error — ask an owner or admin to review the surface with you.

​
Read a citations row

Each ungrounded-turn row carries the fields you need to place the turn:
  • Outcome — the turn’s outcome label as the runtime scored it.
  • Source count — how many knowledge chunks retrieval pulled into context. A turn that retrieved four chunks and cited none is a different problem from one that retrieved zero.
  • Confidence — the highest similarity score across the retrieved chunks, cross-referenced with the citation verdict. A turn with no marker and a low confidence scores below your low-confidence threshold — the totals strip counts these separately as the highest-risk bucket, and the banner tells you how many to triage first.
  • Timestamp and latency — when the turn ran and its time-to-first-byte, so you can correlate a coverage drop with a deploy or a model switch.

​
When citations matter

Three recurring jobs:
  1. Debug a wrong answer. The caller got a bad reply — open the audit, find the turn (or a window full of similar turns), and check whether the answer had retrieval hits it ignored, or no retrieval at all.
  2. Verify the intended source. You attached a knowledge base expecting the agent to use it. The per-agent row tells you whether citations are actually landing; the trend tells you whether that changed at a deploy.
  3. Catch drift early. A dropping citation-coverage rate on live traffic is the cheapest early-warning signal an agent gives you — before the first escalation report, not after.

​
How grounding is computed

The pipeline runs in three stages per turn, and a citation row is the record of all three:
  1. Retrieve — the runtime pulls the top knowledge chunks into context. Each chunk carries its knowledge-base id, the document it came from, the snippet span, and a cosine similarity score.
  2. Score — the evaluator compares the [KB-N] markers in the reply against that retrieval window and grades the turn one of four verdicts: ok (cited a valid marker), no_marker (retrieval fired, nothing cited), orphan_marker (cited a marker that points past what retrieval found — a hallucinated citation), or empty_retrieval (no retrieval and no markers, counted separately so a greeting does not read as an ungrounded answer).
  3. Render — the audit surfaces the verdict, the cited chunk indexes, and the retrieval metadata. The verbatim transcript stays behind the conversation surface; the audit row carries pointers, not the response text.
A null confidence on a turn means no retrieval pass ran — read that as “knowledge grounding did not happen this turn,” not as a score of zero.

​
Fix a bad citation

An ungrounded turn row points at the agent and the window; the fix happens in the knowledge base.
  1. From the dashboard, open Agents → your agent → Knowledge base and find the document whose retrieval window was weak — or missing entirely.
  2. If retrieval returned nothing relevant (empty_retrieval), the knowledge is absent or pending approval. Add or approve the document, and the chunks embed within the minute.
  3. If retrieval returned chunks the model ignored (no_marker), tighten the chunk itself — retrieval order follows the similarity score, and a sharper snippet span wins the citation. The response shape a row carries back — knowledge_base_id, document title, snippet span — is what you match against.
  4. If the model invented markers (orphan_marker), that is a hallucinated citation; the per-turn audit is the only place those markers are visible, because the conversation view drops them. Re-check the cited document content and the agent’s citation directive.
For high-trust lines (healthcare, financial disclosure, regulated workflows) you can go further: opt the agent into requiring citations, so a retrieval turn that emits no well-formed marker fails its output guardrail instead of delivering an ungrounded answer. The audit remains observational; enforcement is a per-agent flag. See Per-turn grounding citations for the verdict model and the flag.

​
Drive the audit over the API

The same data is available programmatically, newest first. Per-agent turn feed (GET /agents/agent_abc123/grounding/citations) — verdicts, cited chunk indexes, and a page-scoped rollup, workspace roles owner/admin/developer or an API key with the agents:read scope:
Each page returns rows (per-turn verdicts and retrieval pointers), summary (grounded vs ungrounded counts and a count per verdict), and next_cursor — pass the cursor back to walk older pages; null ends the feed. limit accepts 1–200 (default 50), and from / to bound the window with ISO-8601 timestamps. Tenant-wide voice rollup (GET /agents/grounding-citations-audit) — the totals, per-agent breakdown, recent ungrounded turns, and daily trend behind the dashboard page. Owner or admin only:
Query parameters: agentId narrows the rollup to one agent; from / to bound the window with ISO-8601 timestamps (otherwise days, 1–365, sizes it); lowConfidenceThreshold moves the confidence cutoff for the low-confidence counters; turnLimit caps the recent-ungrounded triage list (1–200). If you see a 400, check that your timestamps parse as ISO-8601.

​
See also