> ## Documentation Index
> Fetch the complete documentation index at: https://docs.orbit.devotel.io/llms.txt
> Use this file to discover all available pages before exploring further.

# Voice Agent Quality Index (VAQI): latency, turn-taking, and barge-in

> Read the live, per-session latency, turn-taking, and barge-in metrics your AI voice agents produce — and tell a cold model from a real regression.

# Voice Agent Quality Index (VAQI)

An AI voice agent can pass every offline eval and still feel slow to live callers. VAQI is the live-session complement to evals: it aggregates per-session quality records your voice agents emit on every call and rolls them up into latency, turn-taking, and barge-in metrics over a time window you pick — 24 hours, 7 days, or 30 days.

Use it to answer "how is the fleet doing right now," not "does this agent build pass its tests."

**Base path:** `/api/v1/voice`

**Authentication:** Clerk session (`Authorization: Bearer <token>`) or API key (`X-API-Key`).

**Scope:** `voice` read. Dashboard page: **Voice → Voice Agent Quality** (available to owners, admins, and developers).

***

## 1. What VAQI measures

Every completed AI voice session produces one record: duration, turn count, the time the agent took to start responding, the silence between caller and agent turns, speech-to-text / LLM / text-to-speech processing time, and how interruption (barge-in) behaved. VAQI returns:

* `sessions` — sessions in the window
* `avg_session_duration_ms` / `avg_turns_per_session`
* `avg_ttfb_ms`, `p95_ttfb_ms` — time-to-first-byte: how fast the agent starts speaking after the caller's turn ends
* `avg_turn_gap_ms`, `p95_turn_gap_ms` — the silent gap between turns across the session
* `avg_stt_ms`, `avg_llm_ms`, `avg_tts_ms` — pipeline stage averages
* `barge_in_rate` — % of turns where the caller interrupted the agent
* `barge_in_success_rate` — % of those interruptions where the agent yielded cleanly
* `daily[]` — per-day breakdown of sessions, average turn gap, and average TTFB

An empty window returns zeros (not an error), and the fall-back when the rollup store is unavailable is the same zero shape — so a healthy-but-quiet tenant and a storage hiccup both read as "no data," never as broken metrics.

## 2. Reading the three KPI families

**Latency (ms)** — TTFB is the number callers feel first. The dashboard flags a warning when the p95 exceeds 1,200 ms. Splitting the pipeline (STT vs LLM vs TTS) tells you which stage drifted: a TTFB spike with a flat STT and TTS but an elevated LLM average points at the model, not the speech pipeline.

**Turn-taking (turns + gaps)** — `avg_turns_per_session` shows whether conversations are real exchanges, and the turn gap measures the dead air between them. The dashboard warns when the p95 turn gap exceeds 1,500 ms. A rising gap with a stable TTFB usually means longer generations later in the session, so check your prompt length and tool call sizes.

**Barge-in (interruptions)** — `barge_in_rate` is the share of turns the caller talks over the agent; `barge_in_success_rate` is the share of those where the agent stopped cleanly. Some interruption is healthy — callers probe and correct. A high rate with a low success rate means callers are fighting a talking-over agent.

Example thresholds that match the dashboard warning markers:

| Metric           | Healthy        | Warning threshold                       |
| ---------------- | -------------- | --------------------------------------- |
| TTFB p95         | under 1,200 ms | over 1,200 ms                           |
| Turn gap p95     | under 1,500 ms | over 1,500 ms                           |
| Barge-in success | trend-stable   | sudden drop on flat interruption volume |

## 3. Picking a window and comparing agents

Choose the narrowest window with enough sessions to make the averages meaningful:

* **24h** while tuning a prompt or model change.
* **7d** for the operating baseline (the default).
* **30d** for trend review.

The rollup covers all voice agents in the tenant. To compare two agents, run each in its own window and read the `daily[]` rows across the days each was live, or route each agent through a comparison window of its own for a clean read. The dashboard page refreshes every 30 seconds, so a shared screen on the 24h window works as a live wallboard.

## 4. Alert thresholds — and how VAQI differs from evals

Wire alerts on the two p95 warnings above; filter small-sample noise by only alerting when the window has enough sessions. Evals and VAQI answer different questions, and you need both:

* **Evals** run offline against recorded or scripted conversations — they answer "did the agent say the right thing."
* **VAQI** is computed from live sessions — it answers "does the agent feel fast and polite to real callers right now."

Treat a failing eval as a correctness gate and a VAQI p95 breach as an operations alert.

## 5. Who reads it

* **Operations** watches p95 TTFB and turn gap and owns the alert thresholds.
* **Agent builders** own the pipeline split (STT/LLM/TTS) and barge-in behavior — a regression maps back to a prompt, model, or voice configuration change.

## 6. Troubleshooting

* **Latency spikes after a deploy**: compare the stage averages against your baseline. If the LLM average moved, the prompt or the model changed; if STT/TTS moved, the speech configuration changed.
* **Cold models**: the first sessions of the day can carry a higher TTFB while the speech and model pipeline warms; widen to the 7d window before you treat it as a regression, and read `daily[]` to see whether today's row is the only offender.
* **High barge-in with low success**: the agent is not yielding — shorten end-of-turn sensitivity or the first response length.
* **All zeros**: either no AI voice sessions completed in the window, or the rollup cache was briefly unavailable. If call volume is running but VAQI still returns zeros, retry — the fall-back is deliberately zero-shaped so the page renders.

## Worked example

`GET /api/v1/voice/vaqi?days=1`

```json theme={null}
{
  "data": {
    "days": 1,
    "sessions": 412,
    "avg_session_duration_ms": 96500,
    "avg_turns_per_session": 14.2,
    "avg_turn_gap_ms": 612,
    "p95_turn_gap_ms": 1380,
    "avg_ttfb_ms": 742,
    "p95_ttfb_ms": 1090,
    "avg_stt_ms": 210,
    "avg_llm_ms": 388,
    "avg_tts_ms": 144,
    "barge_in_rate": 6.8,
    "barge_in_success_rate": 91.3,
    "daily": [
      { "date": "2026-08-31", "sessions": 412, "avg_turn_gap_ms": 612, "avg_ttfb_ms": 742 }
    ]
  }
}
```

Reading it: LLM time dominates TTFB (388 of 742 ms), p95 TTFB is under the 1,200 ms warning marker, and a 6.8% interruption rate with a 91% clean-yield rate is healthy.

### What the dashboard panel shows

**Voice → Voice Agent Quality** renders the same rollup: a sessions, duration, TTFB (avg/p95), and turn-gap (avg/p95) KPI row that highlights in a warning colour past the thresholds; a pipeline split (STT / LLM / TTS); barge-in rate and barge-in success cards; and the per-day breakdown table. A window switcher (24h / 7d / 30d) is part of the URL, so you can deep-link a specific window.

***

Related: [AI agent evals](/agents/continuous-production-evals) · [Call QA scorecards](/voice/call-qa-scorecards) · [AI agent outbound calls](/voice/ai-agent-outbound-calls)