> ## Documentation Index
> Fetch the complete documentation index at: https://docs.orbit.devotel.io/llms.txt
> Use this file to discover all available pages before exploring further.

# Inspect grounding citations on an AI agent

> Read the citations panel, grade ungrounded turns, trace a bad citation back to its knowledge source, and drive the same audit over the API.

# Inspect grounding citations on an AI agent

When an AI agent answers out of your knowledge base, every turn records which
knowledge chunks the model retrieved and which of those it actually cited in
the reply. The grounding-citations surface is where you read that record —
per turn, per agent, and across the whole tenant. Use it when a wrong answer
needs a source-level explanation, or when you want to verify the agent is
standing on the knowledge you attached and not improvising.

This guide walks the dashboard surface (**Agents → RAG Grounding
Citations**), the verdict model behind each row, and the API endpoints that
back it. The per-turn verdicts concept is covered in depth in
[Per-turn grounding citations](/agents/grounding-citations); this page is the
operator walkthrough for the audit surface itself.

## Open the page

In the dashboard, open **Agents** and pick **RAG Grounding Citations** in the
agents group. The page aggregates every voice-agent turn in the selected
window into four blocks:

* **Totals strip** — voice turns, the share of turns where retrieval fired,
  citation coverage (turns whose reply carried at least one citation marker),
  and the no-retrieval share.
* **Recent ungrounded turns** — the newest turns where retrieval fired but
  the reply carried no citation marker, cross-referenced with retrieval
  confidence when the post-hoc scoring pass ran. This is the triage list.
* **Daily trend** — voice-turn volume per day with the grounded share filled
  in, so a coverage dip lines up with the day it started.
* **By agent** — citation coverage and retrieval share per voice agent, so a
  single misbehaving agent is not hidden inside a healthy fleet average.

Access is owner or admin only, because the rollup spans the whole tenant's
grounding behaviour. A member role sees a permission message rather than an
error — ask an owner or admin to review the surface with you.

## Read a citations row

Each ungrounded-turn row carries the fields you need to place the turn:

* **Outcome** — the turn's outcome label as the runtime scored it.
* **Source count** — how many knowledge chunks retrieval pulled into context.
  A turn that retrieved four chunks and cited none is a different problem
  from one that retrieved zero.
* **Confidence** — the highest similarity score across the retrieved chunks,
  cross-referenced with the citation verdict. A turn with no marker and a
  low confidence scores below your low-confidence threshold — the totals
  strip counts these separately as the highest-risk bucket, and the banner
  tells you how many to triage first.
* **Timestamp and latency** — when the turn ran and its time-to-first-byte,
  so you can correlate a coverage drop with a deploy or a model switch.

## When citations matter

Three recurring jobs:

1. **Debug a wrong answer.** The caller got a bad reply — open the audit,
   find the turn (or a window full of similar turns), and check whether the
   answer had retrieval hits it ignored, or no retrieval at all.
2. **Verify the intended source.** You attached a knowledge base expecting
   the agent to use it. The per-agent row tells you whether citations are
   actually landing; the trend tells you whether that changed at a deploy.
3. **Catch drift early.** A dropping citation-coverage rate on live traffic
   is the cheapest early-warning signal an agent gives you — before the first
   escalation report, not after.

## How grounding is computed

The pipeline runs in three stages per turn, and a citation row is the record
of all three:

1. **Retrieve** — the runtime pulls the top knowledge chunks into context.
   Each chunk carries its knowledge-base id, the document it came from, the
   snippet span, and a cosine similarity score.
2. **Score** — the evaluator compares the `[KB-N]` markers in the reply
   against that retrieval window and grades the turn one of four verdicts:
   `ok` (cited a valid marker), `no_marker` (retrieval fired, nothing cited),
   `orphan_marker` (cited a marker that points past what retrieval found — a
   hallucinated citation), or `empty_retrieval` (no retrieval and no markers,
   counted separately so a greeting does not read as an ungrounded answer).
3. **Render** — the audit surfaces the verdict, the cited chunk indexes, and
   the retrieval metadata. The verbatim transcript stays behind the
   conversation surface; the audit row carries pointers, not the response
   text.

A **null** confidence on a turn means no retrieval pass ran — read that as
"knowledge grounding did not happen this turn," not as a score of zero.

## Fix a bad citation

An ungrounded turn row points at the agent and the window; the fix happens in
the knowledge base.

1. From the dashboard, open **Agents → your agent → Knowledge base** and find
   the document whose retrieval window was weak — or missing entirely.
2. If retrieval returned nothing relevant (`empty_retrieval`), the knowledge
   is absent or pending approval. Add or approve the document, and the
   chunks embed within the minute.
3. If retrieval returned chunks the model ignored (`no_marker`), tighten the
   chunk itself — retrieval order follows the similarity score, and a sharper
   snippet span wins the citation. The response shape a row carries back —
   `knowledge_base_id`, document title, snippet span — is what you match
   against.
4. If the model invented markers (`orphan_marker`), that is a hallucinated
   citation; the per-turn audit is the only place those markers are visible,
   because the conversation view drops them. Re-check the cited document
   content and the agent's citation directive.

For high-trust lines (healthcare, financial disclosure, regulated workflows)
you can go further: opt the agent into requiring citations, so a retrieval
turn that emits no well-formed marker fails its output guardrail instead of
delivering an ungrounded answer. The audit remains observational; enforcement
is a per-agent flag. See [Per-turn grounding
citations](/agents/grounding-citations) for the verdict model and the flag.

## Drive the audit over the API

The same data is available programmatically, newest first.

**Per-agent turn feed (`GET /agents/agent_abc123/grounding/citations`)** —
verdicts, cited chunk indexes, and a page-scoped
rollup, workspace roles owner/admin/developer or an API key with the
`agents:read` scope:

```bash theme={null}
curl "https://api.orbit.devotel.io/api/v1/agents/agent_abc123/grounding/citations?limit=50" \
  -H "X-API-Key: dv_live_sk_..."
```

Each page returns `rows` (per-turn verdicts and retrieval pointers),
`summary` (grounded vs ungrounded counts and a count per verdict), and
`next_cursor` — pass the cursor back to walk older pages; `null` ends the
feed. `limit` accepts 1–200 (default 50), and `from` / `to` bound the window
with ISO-8601 timestamps.

**Tenant-wide voice rollup (`GET /agents/grounding-citations-audit`)** —
the totals, per-agent breakdown, recent ungrounded turns, and daily trend
behind the dashboard page. Owner or admin only:

```bash theme={null}
curl "https://api.orbit.devotel.io/api/v1/agents/grounding-citations-audit?days=30" \
  -H "X-API-Key: dv_live_sk_..."
```

Query parameters: `agentId` narrows the rollup to one agent; `from` / `to`
bound the window with ISO-8601 timestamps (otherwise `days`, 1–365, sizes
it); `lowConfidenceThreshold` moves the confidence cutoff for the
low-confidence counters; `turnLimit` caps the recent-ungrounded triage list
(1–200). If you see a 400, check that your timestamps parse as ISO-8601.

## See also

* [Per-turn grounding citations](/agents/grounding-citations) — the verdict
  model, the per-turn `grounding` block, and citation enforcement
* [The risk/verdict model](/concepts/risk-verdict-model) — how graded
  verdicts drive the audit surfaces
* [Build and maintain an AI knowledge
  base](/guides/knowledge-base-lifecycle) — the document lifecycle behind the
  cited sources
* [Auto-draft help articles](/guides/kb-auto-draft) — close the loop when
  the audit shows the knowledge is missing