​

Voice Agent Quality Index (VAQI)

​

1. What VAQI measures

• sessions — sessions in the window
• avg_session_duration_ms / avg_turns_per_session
• avg_ttfb_ms, p95_ttfb_ms — time-to-first-byte: how fast the agent starts speaking after the caller’s turn ends
• avg_turn_gap_ms, p95_turn_gap_ms — the silent gap between turns across the session
• avg_stt_ms, avg_llm_ms, avg_tts_ms — pipeline stage averages
• barge_in_rate — % of turns where the caller interrupted the agent
• barge_in_success_rate — % of those interruptions where the agent yielded cleanly
• daily[] — per-day breakdown of sessions, average turn gap, and average TTFB

​

2. Reading the three KPI families

​

3. Picking a window and comparing agents

• 24h while tuning a prompt or model change.
• 7d for the operating baseline (the default).
• 30d for trend review.

​

4. Alert thresholds — and how VAQI differs from evals

• Evals run offline against recorded or scripted conversations — they answer “did the agent say the right thing.”
• VAQI is computed from live sessions — it answers “does the agent feel fast and polite to real callers right now.”

​

5. Who reads it

• Operations watches p95 TTFB and turn gap and owns the alert thresholds.
• Agent builders own the pipeline split (STT/LLM/TTS) and barge-in behavior — a regression maps back to a prompt, model, or voice configuration change.

​

6. Troubleshooting

• Latency spikes after a deploy: compare the stage averages against your baseline. If the LLM average moved, the prompt or the model changed; if STT/TTS moved, the speech configuration changed.
• Cold models: the first sessions of the day can carry a higher TTFB while the speech and model pipeline warms; widen to the 7d window before you treat it as a regression, and read daily[] to see whether today’s row is the only offender.
• High barge-in with low success: the agent is not yielding — shorten end-of-turn sensitivity or the first response length.
• All zeros: either no AI voice sessions completed in the window, or the rollup cache was briefly unavailable. If call volume is running but VAQI still returns zeros, retry — the fall-back is deliberately zero-shaped so the page renders.

​

Read VAQI from the API

curl -X GET "https://api.orbit.devotel.io/api/v1/voice/vaqi?days=30" \
  -H "X-API-Key: $ORBIT_API_KEY"

import { Orbit } from "@devotel/sdk-node";

const client = new Orbit({ apiKey: process.env.ORBIT_API_KEY });
const rollup = await client.voice.vaqi.get({ days: 30 });

console.log(rollup.data.sessions, rollup.data.p95_ttfb_ms);
for (const day of rollup.data.daily) {
  console.log(day.date, day.avg_ttfb_ms);
}

from orbit_sdk import OrbitClient

client = OrbitClient.from_env()  # reads ORBIT_API_KEY
rollup = client.request("GET", "/voice/vaqi", params={"days": "30"})
print(rollup["data"]["sessions"], rollup["data"]["p95_ttfb_ms"])

{
  "data": {
    "days": 30,
    "sessions": 412,
    "avg_session_duration_ms": 96500,
    "avg_turns_per_session": 14.2,
    "avg_turn_gap_ms": 612,
    "p95_turn_gap_ms": 1380,
    "avg_ttfb_ms": 742,
    "p95_ttfb_ms": 1090,
    "avg_stt_ms": 210,
    "avg_llm_ms": 388,
    "avg_tts_ms": 144,
    "barge_in_rate": 6.8,
    "barge_in_success_rate": 91.3,
    "daily": [
      { "date": "2026-09-20", "sessions": 412, "avg_turn_gap_ms": 612, "avg_ttfb_ms": 742 }
    ]
  }
}

​

Fields and dashboard thresholds

​

Worked example

{
  "data": {
    "days": 1,
    "sessions": 412,
    "avg_session_duration_ms": 96500,
    "avg_turns_per_session": 14.2,
    "avg_turn_gap_ms": 612,
    "p95_turn_gap_ms": 1380,
    "avg_ttfb_ms": 742,
    "p95_ttfb_ms": 1090,
    "avg_stt_ms": 210,
    "avg_llm_ms": 388,
    "avg_tts_ms": 144,
    "barge_in_rate": 6.8,
    "barge_in_success_rate": 91.3,
    "daily": [
      { "date": "2026-08-31", "sessions": 412, "avg_turn_gap_ms": 612, "avg_ttfb_ms": 742 }
    ]
  }
}

​

What the dashboard panel shows