> ## Documentation Index
> Fetch the complete documentation index at: https://docs.orbit.devotel.io/llms.txt
> Use this file to discover all available pages before exploring further.

# Quality Management API

> Score agent calls with weighted scorecards, run calibration sessions, roll up org-wide quality, and drive agent gamification for your contact center.

# Quality Management API

Run a contact-center quality program end to end: author weighted evaluation forms, score agent calls (or let the AI auto-score them and override when needed), run blind calibration sessions to measure reviewer agreement, read an org-wide quality rollup across voice and inbox, and rank agents on a configurable leaderboard.

**Base path:** `/api/v1/quality`

**Authentication:** API key (`X-API-Key`) or session JWT.

**Roles:** the reviewer endpoints (forms, scoring, calibration, gamification leaderboards, and the org summary) require the `owner`, `admin`, or `supervisor` role. An agent can read their own evaluations (`GET /evaluations` returns only their own rows), acknowledge or appeal an evaluation made about them, and read their own gamification card (`GET /gamification/me`).

All data is scoped to the calling tenant. Validation failures return `422` with a `VALIDATION_ERROR` body; see [Error codes](/api-reference/error-codes).

<Note>
  This is the contact-center **Quality Management** resource — evaluation
  scorecards, calibration, and agent gamification. It is unrelated to the
  per-call **Voice AI Quality** (VAQI) metrics documented under
  [`/api/v1/voice/intelligence`](/api-reference/voice). One scores how a human
  agent handled a call; the other scores call audio and AI-agent behavior.
</Note>

## Evaluation forms

A form defines the rubric a reviewer scores against: a list of weighted sections, each holding weighted criteria. Weights are bare numbers and are normalized within their section and across the form, so you can author them as percentages or as plain relative weights. Each criterion has a `max_score` (default `100`); reviewers submit a raw `0..max_score` value per criterion and the server rolls everything up to a single `0..100` total. Mark a section `auto_fail` to force the whole evaluation to `0` when any criterion in it scores zero (the compliance-gate pattern).

| Method  | Path                                    | Purpose                            |
| ------- | --------------------------------------- | ---------------------------------- |
| `POST`  | `/api/v1/quality/evaluation-forms`      | Create an evaluation form          |
| `GET`   | `/api/v1/quality/evaluation-forms`      | List forms (filter by `is_active`) |
| `GET`   | `/api/v1/quality/evaluation-forms/{id}` | Get a form                         |
| `PATCH` | `/api/v1/quality/evaluation-forms/{id}` | Update a form                      |

Set `sample_rate_pct` (0–100) on a form to have the auto-sampler queue a weighted share of each agent's calls as pending evaluations for a reviewer to score. `0` (the default) keeps the form manual-only.

```json Example form definition theme={null}
{
  "name": "Inbound Support QA",
  "sample_rate_pct": 10,
  "definition": {
    "sections": [
      {
        "id": "compliance",
        "label": "Compliance",
        "weight": 1,
        "auto_fail": true,
        "criteria": [
          { "id": "pii", "label": "Verified identity before disclosure", "weight": 1 }
        ]
      },
      {
        "id": "soft-skills",
        "label": "Soft skills",
        "weight": 2,
        "criteria": [
          { "id": "empathy", "label": "Showed empathy", "weight": 1 },
          { "id": "clarity", "label": "Clear next steps", "weight": 1 }
        ]
      }
    ]
  }
}
```

## Evaluations

An evaluation scores one call for one agent against one form. The reviewer submits a `scores` map of `criterion_id → 0..max_score`; the server re-derives `total_score` from the form weights, so a client can never set the total directly.

| Method | Path                                           | Purpose                                    |
| ------ | ---------------------------------------------- | ------------------------------------------ |
| `POST` | `/api/v1/quality/evaluations`                  | Score a call against a form                |
| `GET`  | `/api/v1/quality/evaluations`                  | List evaluations (keyset paginated)        |
| `POST` | `/api/v1/quality/evaluations/{id}/acknowledge` | Agent acknowledges their evaluation        |
| `POST` | `/api/v1/quality/evaluations/{id}/appeal`      | Agent appeals with a written note          |
| `POST` | `/api/v1/quality/evaluations/{id}/resolve`     | Reviewer closes an appealed evaluation     |
| `POST` | `/api/v1/quality/evaluations/{id}/claim-score` | Reviewer scores a pending auto-sampled row |
| `POST` | `/api/v1/quality/evaluations/{id}/override`    | Reviewer corrects an AI auto-scored row    |

### Status lifecycle

An evaluation moves through `pending → acknowledged` or `pending → appealed → resolved`:

* The evaluated agent (and only that agent) can `acknowledge` or `appeal` a `pending` or `acknowledged` evaluation. An appeal requires an `appeal_note`.
* A reviewer closes an `appealed` evaluation with `resolve`. Resolving closes the appeal; it does not regrade the score.

### Listing and pagination

`GET /evaluations` orders newest-first and uses an opaque keyset cursor. Read `next_cursor` from a response and pass it back as the `cursor` query parameter to fetch the next page; a `null` cursor means there are no more rows. Filter with `agent_id`, `call_id`, `reviewer_id`, `status`, `auto_scored`, and `flagged_for_review`. Agents may only list their own evaluations — the `agent_id` filter is forced to the caller for non-reviewer roles.

### Auto-sampling and AI auto-scoring

Two kinds of system-created rows surface in the list with a distinct `reviewer_id` and may be claimed or corrected:

* **Auto-sampled** rows are queued un-scored (`total_score` `0`) by a form's `sample_rate_pct`. A reviewer fills in the real per-criterion scores with `claim-score`; the server re-derives the total and stamps the reviewer. Only a `pending`, still-unclaimed auto-sampled row is eligible — claiming a human-authored row returns `409`.
* **AI auto-scored** rows carry `auto_scored: true`, and those below your tenant's flag threshold carry `flagged_for_review: true`. A reviewer corrects one with `override`; the corrected total is re-derived, the row is stamped with the reviewer, and `flagged_for_review` is cleared. Only a `pending`, not-yet-corrected AI row is eligible.

## Calibration

A calibration session pins one call and one form and invites a roster of reviewers who each score the call **blind** — they cannot see another reviewer's score or the AI's score until they submit their own. Once two or more reviewers have submitted, the report computes per-criterion variance, overall agreement, each reviewer's delta from consensus, and the AI's drift from that consensus. Calibration scores are practice rows; they never feed the agent-facing scorecard, ack/appeal flow, or leaderboard.

| Method | Path                                       | Purpose                                             |
| ------ | ------------------------------------------ | --------------------------------------------------- |
| `POST` | `/api/v1/quality/calibrations`             | Create a calibration session                        |
| `GET`  | `/api/v1/quality/calibrations`             | List sessions (filter by `status`, `call_id`)       |
| `GET`  | `/api/v1/quality/calibrations/{id}`        | Get a session (roster, counts, your own score only) |
| `POST` | `/api/v1/quality/calibrations/{id}/scores` | Submit your blind score                             |
| `GET`  | `/api/v1/quality/calibrations/{id}/report` | Read the agreement report                           |
| `POST` | `/api/v1/quality/calibrations/{id}/close`  | Close the session                                   |

The session detail and the report enforce the blind rule server-side: an invited reviewer who has not yet submitted gets `403` from the report until they score. Each reviewer may submit once per session (a second attempt returns `409`). If the call already has an AI auto-scored evaluation against the same form, it is seeded read-only as a drift baseline.

## Quality summary

A cross-channel rollup of LLM-judged conversation outcomes across both voice and inbox, for a supervisor dashboard.

| Method | Path                      | Purpose                 |
| ------ | ------------------------- | ----------------------- |
| `GET`  | `/api/v1/quality/summary` | Org-wide quality rollup |

Pass `days` (1–90, default 30) and `conv_type` (`inbox`, `voice`, or `all`). The response returns overall totals (scored count, pass rate, average confidence), a per-channel breakdown, the top rubrics by evaluation volume with their pass rates, and a short list of recent low-confidence failures for triage.

## Gamification

Rank agents on points and badges computed on demand from existing QA, call, and CSAT aggregates — no extra setup. The board is derived for a window and an optional queue or agent allow-list against a configurable ruleset.

| Method | Path                                       | Purpose                                   |
| ------ | ------------------------------------------ | ----------------------------------------- |
| `GET`  | `/api/v1/quality/gamification/config`      | Default point rules and badge definitions |
| `GET`  | `/api/v1/quality/gamification/me`          | An agent's own points, badges, and rank   |
| `POST` | `/api/v1/quality/gamification/leaderboard` | Compute the ranked leaderboard            |

Scope a leaderboard with `period` (`day`, `week`, `month`) or an explicit `from`/`to` window, and optionally a `queue_id` or `agent_ids` list. Owner and admin callers see all agents; a supervisor sees only the agents in the queues they supervise, and requesting a queue they do not supervise returns `403`. Pass `point_rules` or `badge_definitions` in the body to preview a tuned ruleset before adopting it. `GET /gamification/me` returns only the calling agent's slice — ranked against the same default board so the agent's rank always matches the supervisor view.

## See also

* [Voice API](/api-reference/voice) — per-call Voice AI Quality (VAQI) metrics
* [Workforce Management API](/api-reference/wfm)
* [Analytics API](/api-reference/analytics)
* [Error codes](/api-reference/error-codes)