Skip to main content

​
Quality Management API

Run a contact-center quality program end to end: author weighted evaluation forms, score agent calls (or let the AI auto-score them and override when needed), run blind calibration sessions to measure reviewer agreement, read an org-wide quality rollup across voice and inbox, and rank agents on a configurable leaderboard. Base path: /api/v1/quality Authentication: API key (X-API-Key) or session JWT. Roles: the reviewer endpoints (forms, scoring, calibration, gamification leaderboards, and the org summary) require the owner, admin, or supervisor role. An agent can read their own evaluations (GET /evaluations returns only their own rows), acknowledge or appeal an evaluation made about them, and read their own gamification card (GET /gamification/me). All data is scoped to the calling tenant. Validation failures return 422 with a VALIDATION_ERROR body; see Error codes.
This is the contact-center Quality Management resource — evaluation scorecards, calibration, and agent gamification. It is unrelated to the per-call Voice AI Quality (VAQI) metrics documented under /api/v1/voice/intelligence. One scores how a human agent handled a call; the other scores call audio and AI-agent behavior.

​
Evaluation forms

A form defines the rubric a reviewer scores against: a list of weighted sections, each holding weighted criteria. Weights are bare numbers and are normalized within their section and across the form, so you can author them as percentages or as plain relative weights. Each criterion has a max_score (default 100); reviewers submit a raw 0..max_score value per criterion and the server rolls everything up to a single 0..100 total. Mark a section auto_fail to force the whole evaluation to 0 when any criterion in it scores zero (the compliance-gate pattern). Set sample_rate_pct (0–100) on a form to have the auto-sampler queue a weighted share of each agent’s calls as pending evaluations for a reviewer to score. 0 (the default) keeps the form manual-only.
Example form definition

​
Evaluations

An evaluation scores one call for one agent against one form. The reviewer submits a scores map of criterion_id → 0..max_score; the server re-derives total_score from the form weights, so a client can never set the total directly.

​
Status lifecycle

An evaluation moves through pending → acknowledged or pending → appealed → resolved:
  • The evaluated agent (and only that agent) can acknowledge or appeal a pending or acknowledged evaluation. An appeal requires an appeal_note.
  • A reviewer closes an appealed evaluation with resolve. Resolving closes the appeal; it does not regrade the score.

​
Listing and pagination

GET /evaluations orders newest-first and uses an opaque keyset cursor. Read next_cursor from a response and pass it back as the cursor query parameter to fetch the next page; a null cursor means there are no more rows. Filter with agent_id, call_id, reviewer_id, status, auto_scored, and flagged_for_review. Agents may only list their own evaluations — the agent_id filter is forced to the caller for non-reviewer roles.

​
Auto-sampling and AI auto-scoring

Two kinds of system-created rows surface in the list with a distinct reviewer_id and may be claimed or corrected:
  • Auto-sampled rows are queued un-scored (total_score 0) by a form’s sample_rate_pct. A reviewer fills in the real per-criterion scores with claim-score; the server re-derives the total and stamps the reviewer. Only a pending, still-unclaimed auto-sampled row is eligible — claiming a human-authored row returns 409.
  • AI auto-scored rows carry auto_scored: true, and those below your tenant’s flag threshold carry flagged_for_review: true. A reviewer corrects one with override; the corrected total is re-derived, the row is stamped with the reviewer, and flagged_for_review is cleared. Only a pending, not-yet-corrected AI row is eligible.

​
Calibration

A calibration session pins one call and one form and invites a roster of reviewers who each score the call blind — they cannot see another reviewer’s score or the AI’s score until they submit their own. Once two or more reviewers have submitted, the report computes per-criterion variance, overall agreement, each reviewer’s delta from consensus, and the AI’s drift from that consensus. Calibration scores are practice rows; they never feed the agent-facing scorecard, ack/appeal flow, or leaderboard. The session detail and the report enforce the blind rule server-side: an invited reviewer who has not yet submitted gets 403 from the report until they score. Each reviewer may submit once per session (a second attempt returns 409). If the call already has an AI auto-scored evaluation against the same form, it is seeded read-only as a drift baseline.

​
Quality summary

A cross-channel rollup of LLM-judged conversation outcomes across both voice and inbox, for a supervisor dashboard. Pass days (1–90, default 30) and conv_type (inbox, voice, or all). The response returns overall totals (scored count, pass rate, average confidence), a per-channel breakdown, the top rubrics by evaluation volume with their pass rates, and a short list of recent low-confidence failures for triage.

​
Gamification

Rank agents on points and badges computed on demand from existing QA, call, and CSAT aggregates — no extra setup. The board is derived for a window and an optional queue or agent allow-list against a configurable ruleset. Scope a leaderboard with period (day, week, month) or an explicit from/to window, and optionally a queue_id or agent_ids list. Owner and admin callers see all agents; a supervisor sees only the agents in the queues they supervise, and requesting a queue they do not supervise returns 403. Pass point_rules or badge_definitions in the body to preview a tuned ruleset before adopting it. GET /gamification/me returns only the calling agent’s slice — ranked against the same default board so the agent’s rank always matches the supervisor view.

​
See also