> ## Documentation Index
> Fetch the complete documentation index at: https://docs.orbit.devotel.io/llms.txt
> Use this file to discover all available pages before exploring further.

# The QM coaching loop: queue, notes, scorecards, effectiveness

> Run the full quality-management coaching loop on digital conversations: triage low-scoring conversations from the supervisor queue, attach coaching notes, have agents acknowledge them, and measure whether the coaching moved response time, resolution rate, and CSAT.

# The QM coaching loop: queue, notes, scorecards, effectiveness

Quality scoring tells you *which* conversations went badly. Coaching is what you do about it. Devotel Orbit ships the whole loop in one surface: a supervisor queue of the conversations the LLM judge flagged, coaching notes attached to a specific conversation with rubric references, agent acknowledgement so the feedback is confirmed read, and a measured before/after report that tells you whether the coaching actually moved the agent's numbers.

This page covers the *asynchronous* coaching loop on digital conversations (chat, email, WhatsApp, social). It is a different surface from the [live whisper coaching in the softphone](/guides/agent-assist-whisper-coaching), which feeds suggestions to a voice agent while a call is still in progress.

**Base path:** `/api/v1/inbox`

**Dashboard surface:** open the **Inbox → Coaching** page; agents read their own numbers on **/me/scorecard**.

***

## The coaching loop

The loop runs on the LLM-judged rubric outcomes that power the [quality-management program](/guides/quality-management-program). Each step maps to an endpoint:

1. **Supervisor triages the queue.** Scored conversations whose average rubric score is below your threshold (and with at least one failed rubric) surface in the coaching queue, lowest score first.
2. **Supervisor writes a coaching note.** The note is attached to a specific conversation, references the failed rubrics, and may carry a suggested action.
3. **Agent acknowledges the note.** The agent the note is addressed to marks it as read. Until acknowledged, the note sits open on the conversation.
4. **Scorecard tracks the agent.** Per-rubric pass rates, a daily trend, and the agent's top failure modes over a window (default 30 days).
5. **Effectiveness is measured.** The report compares the agent's response time, resolution rate, and CSAT in equal-length windows before and after the coaching event, and classifies each KPI as improved, regressed, or flat.

Response time is "lower is better"; resolution rate and CSAT are "higher is better." The report bakes polarity in so a falling response time reads as an improvement, not a regression.

***

## Triage the coaching queue

`GET /inbox/coaching/queue` returns conversations that scored below your threshold, ordered by lowest average score — so the worst come up first.

```bash theme={null}
curl -X GET "https://api.orbit.devotel.io/api/v1/inbox/coaching/queue?threshold=0.5&limit=20" \
  -H "X-API-Key: dv_live_sk_your_key_here"
```

Query parameters:

| Parameter   | Meaning                                                            | Default |
| ----------- | ------------------------------------------------------------------ | ------- |
| `threshold` | Average rubric score below this value flags the conversation (0–1) | `0.5`   |
| `limit`     | Page size (max 50)                                                 | `20`    |
| `cursor`    | Pagination cursor from the previous page's `next_cursor`           | —       |
| `conv_type` | `inbox`, `voice`, or `all`                                         | `all`   |

Each queue entry carries the conversation id, the agent it is assigned to, the average and minimum rubric scores, the failed/total rubric counts, and when it was last scored. A tenant with no QM scoring provisioned yet gets an empty queue with a 200 — not an error.

***

## Attach and list coaching notes

A supervisor attaches a note to a specific conversation with `POST /inbox/conversations/:id/coaching-notes`:

```bash theme={null}
curl -X POST "https://api.orbit.devotel.io/api/v1/inbox/conversations/conv_abc123/coaching-notes" \
  -H "X-API-Key: dv_live_sk_your_key_here" \
  -H "Content-Type: application/json" \
  -d '{
    "agent_id": "user_2sXkV9pQ",
    "note": "First reply asked for the account number but skipped identity verification before disclosing the balance. Rework this thread and confirm the fixed phrasing.",
    "rubric_ids": ["rub_verify_identity"],
    "suggested_action": "Review the identity-verification rubric walkthrough before your next shift."
  }'
```

The body's `agent_id` must match the agent the conversation is assigned to — a mismatch returns `422 AGENT_MISMATCH`, which keeps a note from being filed against the wrong teammate. Up to 10 rubric references attach to a note; the note text is capped at 5,000 characters.

List a conversation's coaching history — newest first — with `GET /inbox/conversations/:id/coaching-notes`:

```bash theme={null}
curl -X GET "https://api.orbit.devotel.io/api/v1/inbox/conversations/conv_abc123/coaching-notes" \
  -H "X-API-Key: dv_live_sk_your_key_here"
```

Owners, admins, and supervisors see every note on the conversation; an agent sees only the notes addressed to them.

***

## Agent acknowledges the note

The agent the note is addressed to marks it read with `PUT /inbox/conversations/:id/coaching-notes/:noteId/acknowledge`:

```bash theme={null}
curl -X PUT "https://api.orbit.devotel.io/api/v1/inbox/conversations/conv_abc123/coaching-notes/cnote_xyz789/acknowledge" \
  -H "X-API-Key: dv_live_sk_agent_key_here"
```

Only the addressed agent can acknowledge; any other caller gets `403 FORBIDDEN`. The response stamps `acknowledged_at` and `acknowledged_by_agent: true`, which supervisors read back from the note list to confirm the feedback landed.

***

## Read the per-agent scorecard

`GET /inbox/agents/:agentId/scorecard` (with `?days=30`) returns the agent's quality scorecard: overall totals, per-rubric pass rates sorted by lowest first, a daily score trend, and the top failure modes.

```bash theme={null}
curl -X GET "https://api.orbit.devotel.io/api/v1/inbox/agents/user_2sXkV9pQ/scorecard?days=30&conv_type=inbox" \
  -H "X-API-Key: dv_live_sk_your_key_here"
```

The `top_failure_modes` block lists the rubrics with a pass rate under 80% — the top of that list is the coaching focus. An agent may read only their own scorecard; owners, admins, and supervisors read any. The agent-facing view is documented in [Read your own QM scorecard](/guides/my-scorecard).

***

## Measure coaching effectiveness

`GET /inbox/agents/:agentId/coaching-effectiveness` is the "did the coaching work" report. It compares three digital KPIs — response time, resolution rate, and CSAT — in equal-length windows before and after a coaching event:

```bash theme={null}
curl -X GET "https://api.orbit.devotel.io/api/v1/inbox/agents/user_2sXkV9pQ/coaching-effectiveness?window_days=14" \
  -H "X-API-Key: dv_live_sk_your_key_here"
```

* **Anchor.** Pass `anchor_at` (ISO 8601) explicitly, or omit it to anchor on the agent's **most recent coaching note**. If the agent has no notes and no anchor is supplied, the route 404s — there is nothing to measure against.
* **Windows.** Symmetric, `window_days` long on each side (default 14). The after window is capped at "now" so a recent note never measures against a future, empty span; the response sets `in_progress: true` when that happens.
* **Per KPI.** Each entry carries `before`, `after`, `delta`, `pct_change`, and a `direction` of `improved` / `regressed` / `flat` / `insufficient_data`. Response time is lower-is-better; resolution rate and CSAT are higher-is-better. A metric either window lacked data for reads as `insufficient_data` instead of a misleading zero-delta.

The report only counts the agent's digital-channel conversations — voice coaching effectiveness on calls is reported separately by the voice coaching-plans surface.

***

## The leaderboard (optional)

`GET /inbox/leaderboard` ranks every scored agent by rubric pass rate over the window:

```bash theme={null}
curl -X GET "https://api.orbit.devotel.io/api/v1/inbox/leaderboard?days=30&limit=25" \
  -H "X-API-Key: dv_live_sk_your_key_here"
```

The leaderboard is **opt-in**: an admin enables it under **Settings → Quality Management**. Until enabled, the route returns 403. Privacy is layered per caller:

* Owners, admins, and supervisors always see real names.
* Other callers see names only when the `leaderboard_show_names` org setting is enabled; otherwise every entry is pseudonymized to a stable `Agent-a1b2c3d4` handle, so trend comparisons still work.
* The caller's own entry is always revealed with their real name.

***

## Role matrix

| Capability                                        | Agent                                | Supervisor | Owner / Admin |
| ------------------------------------------------- | ------------------------------------ | ---------- | ------------- |
| Read own scorecard and own coaching effectiveness | Yes                                  | Yes        | Yes           |
| Read the coaches queue                            | —                                    | Yes        | Yes           |
| Create coaching notes                             | —                                    | Yes        | Yes           |
| List coaching notes on a conversation             | Own notes only                       | All notes  | All notes     |
| Acknowledge a coaching note                       | Addressed notes only                 | —          | —             |
| Read the leaderboard                              | Pseudonymized unless names are shown | Real names | Real names    |
| Enable the leaderboard / show names               | —                                    | —          | Yes           |

Every capability in this page is a tenant-owned control: enable the leaderboard, set your queue threshold, and scope who sees names from your own organization settings. Nothing in the coaching loop sends anything to a customer — notes and scorecards live inside your workspace.