Skip to main content

​
QA evaluations and the performance leaderboard

The Quality dashboard pairs two surfaces that run on one scored-evaluations pipeline: Quality → Evaluations, where reviewers author weighted scorecards against calls, and Quality → Leaderboard, a ranked team view that converts QA scores, call volume, and customer satisfaction into points and badges. This page explains both, how the evaluation statuses roll up into leaderboard metrics, and where each surface belongs in a coaching loop.

​
Evaluations: structured call reviews

An evaluation grades one agent’s call against an evaluation form — the weighted rubric a supervisor authors. The pipeline has four pieces:
  1. Forms. A form is sections, criteria, and weights. Each criterion is scored 0–100 raw; section weights and form weights are normalised server-side to a 0–100 total_score, so a client cannot inflate the rollup. A form marked auto_fail forces the total to zero when any one of its criteria scores zero — the classic “compliance” section pattern (for example, a zero on “PII disclosure” fails the whole call). Set sample_rate_pct above 0 to have the platform auto-queue a share of each agent’s calls for human review; 0 keeps the form manual-only.
  2. Authoring. A reviewer (owner, admin, or supervisor) scores a call against an active form. The evaluation lands pending for the agent to react to. Agents can also score their own calls as self-assessments; a self-evaluation reads acknowledged immediately and is excluded from official averages (see below).
  3. The agent’s response. The evaluated agent acknowledges the score, or appeals it with a written note. An appeal moves the status to appealed; a supervisor then resolves it. Resolving closes the appeal — it does not regrade the call.
  4. AI auto-scoring with human override. With auto-scoring enabled, a background pass grades every completed call against your active form and writes it as an evaluation. Scores under your flag threshold (default 70/100) are marked for human review. A reviewer can correct an AI-scored row (override) or pick up an un-scored queued sample (claim-score); either way the weighted total is re-derived from the form’s weights.
All editing happens from the Quality → Evaluations dashboard page. A low QA score can also auto-generate a coaching assignment — an enrolment onto a coaching plan, not just a slack alert — so a weak evaluation kicks off remediation without a supervisor having to remember.

​
Leaderboard: the ranked team view

The Quality → Leaderboard page scores every queue agent on a tuned mix of numeric metrics over a lookback window you choose (day, week, month, or an explicit range). The default points rules reward:
  • Calls handled — volume of completed calls the agent answered.
  • Average QA score — the mean of graded evaluations, scaled up to 60 points so quality outweighs pure volume.
  • Average CSAT — survey responses on the agent’s calls, scaled to 30 points.
  • Efficient handle time — a fixed bonus when the agent’s average handle time is under the target.
Threshold badges mark the top performer, podium finishers, and category leaders. The default ruleset is balanced so no single behaviour dominates; supervisors preview tuned rules against the same board before adopting them. Scope controls matter for team reviews:
  • Supervisors see only the queues they are mapped to — a board scoped to a queue is automatically limited to that supervisor’s teams; a queue outside their scope returns 403.
  • You can narrow a board by queue_id or an explicit agent allow-list, so the board compares a team rather than the whole roster.
  • Agents read their own points, badges, and rank via a self-scoped view computed against the full org and the same default rules — an agent’s rank can never disagree with the supervisor’s board.

​
How scorecards become leaderboard metrics

The leaderboard reads, it does not re-grade. It aggregates per agent over your window, and only official, graded evaluations count toward the QA average:
  • A self-evaluation (the agent scoring their own call) is excluded from the official averages and the leaderboard. Otherwise an agent could inflate their own ranking by scoring themselves — self-assessments feed coaching and calibration instead.
  • An un-scored placeholder queued by the auto-sampler or the CSAT trigger (a 0-score row awaiting a human) is excluded. Counting a zero in the denominator would drag an agent’s average below their true graded score. Once a reviewer claims and scores it, the row reads as the reviewer’s own and joins the official totals.
  • An AI auto-scored evaluation counts the same way a human one does — auto-scored rows are not special-cased in the rollup; the reviewer-override path corrects them in place.
The other two inputs come from what the platform already captures: completed answered calls and talk time for the volume and handle-time metrics, and CSAT survey responses tied to the agent’s calls for the satisfaction metric. No extra ingestion or per-tenant tables are created — the board is computed on demand when you load the page or call the API.

​
Where each surface belongs in a coaching loop

Use the surfaces for different parts of the loop, and pick the right sibling surface for the channel:
  • Evaluations (this page) is the structured review queue — the deterministic rubric a supervisor audits. It covers calls; for the queue of digital conversations (chat, email, WhatsApp, social) and coaching notes on them, use the Inbox coaching loop, which reads the LLM-judged conversation rubric.
  • Leaderboard (this page) is the team-watching view — a fair comparison across agents, scoped to a queue or the org, over a fixed window. Put it on a supervisor’s weekly agenda.
  • Coaching notes and voice live assistance sit alongside: an appeal resolved, a note attached, a live whisper suggestion issued — each a different tool for the same goal of moving the agent’s next evaluation.
A healthy loop looks like: evaluation graded → agent acknowledges or appeals → score lands on the leaderboard → low scores auto-trigger a coaching assignment → next week’s evaluations move.

​
Example: weekly QA sampling workflow

A concrete pattern supervisors run, sized for a mid-sized voice team:
  1. Monday. The supervisor reviews the leaderboard for last week — identifies the bottom decile by QA score and the bottom decile by CSAT, since volume alone is a weak predictor of quality.
  2. Each day. Auto-scoring grades every call. The supervisor spot-audits only flagged rows (those under the 70-point review threshold) and claims a handful of auto-sampled rows per day to keep human coverage on a fixed share of each agent’s calls.
  3. The agent’s part. An agent who disagrees with a score appeals it on the spot; the supervisor sees the pending appeal and resolves it by Friday.
  4. Friday. The supervisor resolves open appeals, checks the week’s board, and turns any agent whose QA average is below the coaching threshold into a coaching plan — the assignment was already generated by the low-score trigger; this is the human review of it.
  5. Monthly. Tune the form: add or re-weight criteria based on which sections actually predict CSAT, and adjust the sample rate if the queue is heavier than reviewers can clear.

​
API surface

The dashboard pages above are backed by the Quality API. Reviewer-scope routes (owner / admin / supervisor) handle forms, evaluations, appeals, overrides, and the leaderboard; agent-scope routes handle acknowledge, appeal, self-evaluation, and the self-scoped leaderboard card. Key routes: The endpoint behaviour this page describes is verified by the platform’s own test suite — including quality/evaluation-scorecard.test.ts in the API package, which asserts the excluded-from-official-averages rules above. The full request/response shapes are in the Quality API reference.