> ## Documentation Index
> Fetch the complete documentation index at: https://docs.orbit.devotel.io/llms.txt
> Use this file to discover all available pages before exploring further.

# Call QA scorecards: rubric grading, trends, and auto-scoring

> Grade any completed call against a five-dimension rubric, follow each agent's composite trend, search your full recording and transcript library, and let the AI scorecard scorer auto-grades every call so supervisors only review a sample.

# Call QA scorecards and auto-scoring

Quality monitoring is the part of a contact-centre stack that usually means buying a separate per-seat module. Orbit gives you the full QA loop where the calls already are — grade any recorded call against a deterministic rubric, follow each agent's rolling composite, find the recordings that matter by keyword instead of scrubbing audio, and let the AI auto-score every call so a supervisor only spot-audits a sample.

This page covers the supervisor-authored scorecard surface (the inline rubric on the call detail page and its API), the per-agent trend view, the transcript and voicemail search library, and the auto-scoring scheduler that grades calls against your tenant's QA evaluation form.

For the richer conversation-grading surface with weighted custom scorecards, appeals, and calibration sessions, see the Quality → QA Evaluations dashboard instead — this page is the call-level rubric that lives on the call detail page.

**Base path:** `/api/v1/voice`

**Authentication:** Clerk session (`Authorization: Bearer <token>`) or API key (`X-API-Key`).

**Scope:** `voice` read for reads; the scorecard upsert is restricted to owners and admins (the dashboard's supervisor persona).

***

## The rubric: five dimensions, 1–5 each

A scorecard grades one call on five dimensions, each on a 1–5 integer scale with an optional per-dimension note (up to 2,000 characters) and an optional overall note (up to 4,000 characters):

| Dimension     | What the reviewer listens for                                                                     |
| ------------- | ------------------------------------------------------------------------------------------------- |
| `greeting`    | Opening — identify the business, use the customer's name where you have it, set the tone.         |
| `hold_notice` | Whether the agent told the caller before putting them on hold and offered a way back.             |
| `empathy`     | Acknowledging what the caller said, mirroring their language, de-escalating when they were upset. |
| `resolution`  | Whether the caller's actual need was met in this call, or a concrete next step was agreed.        |
| `closing`     | Wrapping up — recap, confirm next steps, offer to help further.                                   |

The reviewer is always the authenticated supervisor making the call — the API never trusts a client-supplied reviewer identity, and every save is written to the audit log with the five dimensions so a quarterly QA review can replay who graded what and when.

Composite is the equal-weighted mean of the five scores — the headline number a team tracks week-over-week.

### Where it lives

* **Call detail page.** Open any call under **Voice → Calls → your-call**. Owners and admins see an inline editor (rubric chips + notes fields). Everyone else with call access sees the persisted rubric read-only — the editor never flashes for non-admins because the backend enforces the same gate.
* **Agent trend.** Below the editor, the same panel charts the answering agent's daily composite average over a 30-day window (customisable 1–90 days), so a supervisor can see a single save move the agent's rolling quality number. Agents can't be mis-attributed — the backend resolves the answering agent from the call's metadata (`agent_user_id`, fallback `user_id`), not from a client-provided field.
* **Recording library.** The unified QA worklist — every recording, its diarised transcript, the latest linked evaluation score, and the recording QC verdict in one query — is documented on [Searchable recording and transcript library (QA)](/voice/recording-library).

### Endpoints

| Operation                                                          | Endpoint                                                                 | Access        |
| ------------------------------------------------------------------ | ------------------------------------------------------------------------ | ------------- |
| Save (upsert — one canonical scorecard per call, last writer wins) | `POST /api/v1/voice/calls/{callId}/qa-scorecard`                         | owner / admin |
| Read one call's scorecard                                          | `GET /api/v1/voice/calls/{callId}/qa-scorecard`                          | `voice:read`  |
| Derive an agent's daily trend                                      | `GET /api/v1/voice/qa-scorecards/trend?agent_user_id={user}&days={1-90}` | `voice:read`  |

The save body accepts each `<dimension>_score` (integer 1–5, required) plus `<dimension>_notes` / `overall_notes` (optional). Re-saving the same call updates the rubric in place and bumps `updated_at` — the upsert key is the call id, so two supervisors re-reviewing the same call converge on one canonical scorecard rather than forking. The first save returns `201`; a re-save returns `200`.

The trend endpoint rolls the scorecards up server-side into daily buckets:

```json theme={null}
{
  "results": [
    {
      "bucket_date": "2026-08-24",
      "reviews": 4,
      "avg_greeting": 4.25,
      "avg_hold_notice": 3.75,
      "avg_empathy": 4.0,
      "avg_resolution": 4.25,
      "avg_closing": 4.0,
      "avg_composite": 4.05
    }
  ],
  "agent_user_id": "user_abc123",
  "days": 30
}
```

`avg_composite` is the equal-weighted mean of the five `avg_*` per-dimension averages — no client-side math needed. Reading a call that has not been reviewed returns `404` with a clear message rather than a generic error, so the dashboard renders "Not yet reviewed" cleanly on a fresh tenant.

***

## Finding the right call: searchable recordings and transcripts

You no longer scrub audio to find the call a QA review needs. Three surfaces index every spoken word into full-text search so you can find recordings by keyword, phrase, or negation:

* **Call transcript search** — `GET /api/v1/voice/calls/search` searches the full post-call speech-to-text transcript stored on `call_logs.metadata.transcript` / `transcript_data.transcript`. Query params: `q` (required, up to 500 chars), `limit` (default 25, max 100), `since` / `until` (ISO time range), `direction` (`inbound` / `outbound`).
* **Voicemail transcript search** — `GET /api/v1/voice/voicemails/search` targets voicemail messages specifically — same query shape, narrower surface (`q`, `limit` up to 50, `since`, `until`).
* **PII redaction applies.** Tenants with transcript redaction enabled can still search — the snippet in each result respects the tenant's redaction mode, so raw caller speech never leaks through a QA surface.

Both endpoints use PostgreSQL websearch syntax on the query — so `billing OR refund`, `"angry customer"` (exact phrase), and `cancellation -reschedule` (with an exclusion) all work. Each result carries the call or message id, the standard from/to/direction columns, a relevance `rank`, and a `ts_headline` `snippet` with `<mark>` tags around matched tokens, ready for inline highlight in a dashboard list.

For the unified QA worklist — every recording class, the finalised diarised transcript segments, the latest linked evaluation score, and the recording QC verdict in one keyset-paginated surface — use the recording library: [Searchable recording and transcript library (QA)](/voice/recording-library).

***

## Auto-scoring: AI grades every call against your QA form

Manual rubric grading covers the calls a supervisor picks. Auto-scoring covers the other 100% — a scheduler grades every completed call against your tenant's active QA evaluation form (the same weighted scorecards you build in **Quality → QA Evaluations**) as soon as the transcript lands. Supervisors then spend their time spot-auditing the AI's work rather than scoring every call from scratch.

How it works:

1. A background sweep (60-second cadence) picks up completed calls that have a transcript and no auto-scored evaluation yet.
2. The transcript (capped at 12,000 characters ≈ 3,000 tokens per call) is scored against each criterion on your active QA evaluation form. The LLM only scores individual criteria on each criterion's own 0..max\_score scale; the weighted rollup is computed server-side deterministically, so a form's section weights and auto-fail sections can't be undermined by LLM arithmetic.
3. The result is written as a normal QA evaluation with the `auto-llm` reviewer id and `auto_scored = true` provenance, so downstream readers and your own "percent auto-scored" reports can tell it from a human-authored score.
4. Scores below a flag-for-review threshold (default **70** out of 100, comparable to the vendor defaults) are marked `flagged_for_review` so the supervisor's "needs human review" queue carries only the borderline calls — the Five9 School of Thought QA pattern.

Turn it on or change the threshold in your organization settings under `qa_autoscore` (set `enabled: true`, optionally `flag_threshold: 0..100`), or through the dashboard in **Settings → QA Auto-Score**. A per-tenant cap and per-call LLM deadline keep grading latency under \~2 minutes from hangup without ever burning the model budget on a stuck call.

> Auto-scored evaluations read exactly like human-authored ones to the QA Evaluations dashboard — the agent can still acknowledge or appeal them, and a supervisor can override the score with a normal human review.