> ## Documentation Index
> Fetch the complete documentation index at: https://docs.orbit.devotel.io/llms.txt
> Use this file to discover all available pages before exploring further.

# Read a voice-eval run in the dashboard

> Walk through the voice-eval run detail page — the run summary header, per-case transcripts and judge scores, per-leg latency splits, the Simulated badge, and the run-level failure banner.

# Read a voice-eval run in the dashboard

A voice-eval run replays a golden set (a named bundle of audio prompts with
expected response topics) against your agent and grades every case with an
LLM judge. The run detail page is where you read that evidence: the
aggregate header tells you whether the run passed overall, and each case
row shows you exactly how the agent answered, how long each pipeline leg
took, and — when something went wrong — why.

This page covers what every block on the run detail page means. For where
runs come from (manual triggers and continuous production sampling), see
[Continuous Production Evals](/agents/continuous-production-evals).

## 1. Where the page lives

Dashboard path: **Agents → your agent → Testing (Evals) → Voice eval runs →
pick a run**. The route is
`/agents/:id/evals/voice-runs/:runId`.

The **Voice eval runs** table on the Evals tab lists every run for the agent
with its status and pass rate. Opening a run loads its full detail —
summary header, cases table, and failure diagnostics — in one page.

## 2. Run summary header

The header card aggregates the run so you can read the verdict before
expanding a single case:

* **Status** — `Pending`, `Running`, `Completed`, or `Failed`.
* **The Simulated pill** — shown on every run (see section 5).
* **Golden set and time** — the golden set id and the run's start (or
  creation) time, for pinning regression deltas against the right baseline.
* **Pass rate** — `X/Y = Z%`: passed cases over total cases. A case passes
  when its LLM-judge score clears the golden set's threshold.
* **P95 latency** — the 95th percentile end-to-end case latency.
* **Cost** — the run's total spend.
* **Cases** — total golden-set cases executed.
* **Providers** — the STT provider, TTS provider, and LLM model the run
  executed with, so latency and score comparisons stay apples-to-apples
  across candidate changes.

## 3. Per-case rows

Below the header, the **Cases** table holds one row per golden case:

* **Case** — the case label (expanded in the drawer).
* **Actual response** — the agent's full reply text.
* **Latency** — end-to-end time from prompt to completed response.
* **Score** — the LLM-judge score, colored green when the case passed and
  red when it failed.

Click a row to expand its drawer. The drawer shows the full evidence for
the case:

* **Expected response topic** — what the golden set says a correct answer
  covers. Compare this against the actual response before you trust the
  judge's score either way.
* **Actual response** — the agent's reply, verbatim.
* **Transcript** — the caller prompt as the STT leg heard it. An empty or
  garbled transcript means the problem started upstream of the LLM.
* **Judge reasoning** — the LLM judge's justification for its score. Read
  this before promoting a candidate: a passing score with thin reasoning
  is not a rollout signal.
* **Recording** — the case's audio, playable inline.
* **Failure reason** — when a case failed to execute (not merely failed its
  score), the reason appears here. see section 6 for how failure text is
  presented safely.

While the run is in `Pending` or `Running`, the table shows **No results
yet — run is still processing**; it fills in case-by-case as the runner
finishes each one. A completed run with an empty cases table means the
golden set was empty or every case was skipped — check the golden set
before rerunning.

## 4. Per-leg latency tiles

Each expanded case splits its end-to-end latency into four tiles:

| Tile           | What it measures                    | A spike here means                                                                          |
| -------------- | ----------------------------------- | ------------------------------------------------------------------------------------------- |
| **STT**        | Speech-to-text on the caller prompt | The transcription leg stalled or the stubbed value dominates (simulated runs).              |
| **LLM**        | The agent's reasoning loop          | The model or a tool call is slow — tweak prompt, model, or tool latency.                    |
| **TTS**        | Synthesizing the reply audio        | The voice synthesis leg is slow; on real (non-stubbed) runs, consider a different provider. |
| **End-to-end** | Prompt-to-reply wall time           | The aggregate the P95 header stat is drawn from.                                            |

Distinguishing the legs matters: an **STT failure** shows a missing or
garbled transcript with a low score, but healthy LLM latency — routing
blame to transcription quality. An **LLM slowdown** shows a clean
transcript and a correct-but-slow response, with the LLM tile carrying the
bulk of the end-to-end number. Tuning them is different work.

## 5. The Simulated badge and the stub warning

In the current rollout, the STT and TTS latency legs are **stubbed** — the
API flags every run with `is_simulated: true` and the dashboard attaches a
**Simulated** pill to the header plus an explanatory notice below it.

The rule: **never treat simulated-run STT/TTS latency numbers as
procurement-grade.** The LLM leg and the end-to-end figure contain real
signal; the STT/TTS tiles are placeholder values until those legs are
executed for real. Quotes, SLAs, or provider comparisons must come from a
run against live providers, not from a simulated run.

The API response carries the same flag (`is_simulated`, `simulated_legs`,
`simulation_notice`) so automation can enforce the same rule — filter
simulated runs out of any numbers shipped to a vendor or regulator.

## 6. The run-level failure banner

When an entire run faults — a case-row write error, an unexpected runner
fault, a crash mid-run — the runner persists a **synthetic run-level failure
case** whose golden case id is `__run__`. The dashboard surfaces that
diagnostic as a red **This run failed** banner on the header card and keeps
the `__run__` case out of the per-case results table (it is not a golden
case; it carries no transcript or score).

Reading the banner:

* **Run failed, no case detail** — the diagnostic shows the run-level
  cause, for example a runner fault or an infrastructure error.
* **Run failed mid-cases** — the banner falls back to the first per-case
  failure reason, so a failed run always names a cause rather than showing
  a bare red badge.
* **Failure text is redacted** — provider and SDK error messages can carry
  stack traces, internal hostnames, IP addresses, file paths, and long
  credential-shaped tokens. Before rendering, the page strips those out
  (paths, IPs, cluster hostnames, hex blobs, and UUIDs are replaced with
  placeholders, and the text is truncated to a safe length). When you are
  diagnosing a failure through the API instead, apply the same treatment
  before pasting error text into tickets or chat.

## 7. API parity

Everything on the page comes from one endpoint, so any view on the
dashboard can be reproduced — or exported — over the API:

```bash theme={null}
curl -X GET "https://api.orbit.devotel.io/api/v1/agents/voice-eval/runs/vevr_abc123" \
  -H "X-API-Key: dv_live_sk_..."
```

The response returns the run object (`status`, `cases_total`,
`cases_passed`, `p95_latency_ms`, `cost_usd`, `stt_provider`,
`tts_provider`, `llm_model`, `is_simulated`, `simulated_legs`,
`simulation_notice`) plus the full case array with transcripts, judge
scores, and per-leg latencies. The run-level failure diagnostic is the
case whose `golden_case_id` equals `__run__`.

Useful companions:

* `GET /api/v1/agents/voice-eval/runs/{id}/regression-delta` — compares a
  run against its prior baseline run for the same agent + golden set and
  reports whether P95 latency or judge score regressed.
* `GET /api/v1/agents/voice-eval/runs?status=failed` — monitor for failed
  runs across all agents.

## See also

* [Continuous Production Evals](/agents/continuous-production-evals) —
  where production-sampled runs come from and how the sampling policy is
  configured.
* [Voice evaluation endpoints](/api-reference/endpoints/agents#voice-evaluation) —
  golden sets, run triggers, and regression deltas in full.
* [Troubleshooting: agent eval-queue failures](/troubleshooting/agent-eval-queue) —
  when a run never moves past `queued`, or a run fails to settle.