> ## Documentation Index
> Fetch the complete documentation index at: https://docs.orbit.devotel.io/llms.txt
> Use this file to discover all available pages before exploring further.

# Per-agent coaching cards and the compliance-flag queue on Quality

> The two Quality rollups a supervisor opens first: per-agent coaching cards (average QA score, flagged share, weakest criteria, open coaching plans, badge chips) and the newest-first compliance-flag queue, with role-scoped visibility and reviewer-override clearing.

# Per-agent coaching cards and the compliance-flag queue

Two read surfaces on **Quality** answer the supervisor's morning question — "who needs coaching, and which low auto-scores do I clear first" — before the leaderboard or trend pages are even opened:

* **Coaching cards** — a per-agent rollup over reviewer-authored evaluations: each agent's average QA score, the share of their evaluations flagged for review, their weakest scorecard criteria, the count of open auto-assigned coaching plans, and short badge chips. Cards are sorted lowest-average-first so the drilling list orders itself.
* **The compliance-flag queue** — a newest-first list of evaluations whose `flagged_for_review` bit is set, so the auto-scorer's below-threshold signals clear in a fixed order. Each row carries `flag_reason: "auto_score_below_threshold"` — the one reason a flag can exist today.

Both are read-only rollups over the evaluation ledger: they never mutate a scored evaluation, and only a human override changes what they report.

## 1. Who reads which card

Access splits into reviewer scope and agent self-scope, and each endpoint applies the split identically:

| Role                       | Coaching cards                                  | Compliance flags |
| -------------------------- | ----------------------------------------------- | ---------------- |
| Owner / admin / supervisor | Every evaluated agent.                          | Entire queue.    |
| Agent                      | Own card, narrowed server-side to their own id. | Own flags only.  |

The self-scope narrowing is enforced at request time — an agent passing another agent's `agent_id` still gets their own card back, never a teammate's. The coaching card is also scoped to what it reports: it contains one agent's own numbers and chips, never the league table. An agent in the self-view never sees a rank or another agent's identity.

## 2. The coaching card

For a reviewer, `days` (1–90, default 30), `agent_id`, and `limit` (default 50, clamped at 50) narrow the rollup. A malformed value falls back to the default instead of erroring, so a dropped query parameter degrades rather than blocks the page. Each returned card carries:

* `agent_id` and `form_id` — the agent and the scorecard form the rollup ran on.
* `evals` — the evaluation count behind the averages.
* `avg_score` — mean QA score over those evaluations, `null` on an empty window.
* `flagged` and `flagged_share` — how many evaluations were flagged for review, and that count as a share of `evals`.
* `open_plans` — coaching plans still active for this agent.
* `weak_criteria` — up to five weakest scorecard criteria, rolled up per agent with the criterion label from the form definition. A criterion counts as weak when its normalised percent falls below 70, ranked by most-often-weak then lowest average.
* `chips` — the badge chips that make the card scannable.

The chip semantics are fixed — three chip types, each readable straight off the card:

| Chip id      | When it fires                             | Chip text                                  |
| ------------ | ----------------------------------------- | ------------------------------------------ |
| `qa_score`   | `avg_score` under your coaching threshold | `Low QA score <avg> (below <threshold>)`   |
| `flag_rate`  | A quarter or more of evaluations flagged  | `<pct>% of evaluations flagged for review` |
| `open_plans` | Any active coaching plans                 | `<n> open coaching plan(s)`                |

In the dashboard these same chips render on **Quality → Leaderboard** as the per-agent coaching card next to the board, so a rank and the drilling context sit on one screen. A chip fires because a threshold tripped, so it is a cue to investigate the cut behind it — a `flag_rate` chip on two evaluations reads differently than the same chip on twenty.

Both endpoints only count **official** evaluations. Excluded before any average: an agent's self-evaluation, un-scored auto-sample and CSAT-trigger placeholder rows, and evaluations that moved into `appealed` or `resolved` (a superseded score drops out of the denominator). A thin window of two graded calls produces the same shape as a dense one — read the chips against `evals` before acting on them.

## 3. The compliance-flag queue

`GET /quality/compliance-flags` takes the same `days` (default 30), `agent_id`, and `limit` (default 50, clamped at 100) parameters. Each row carries `evaluation_id`, `agent_id`, `form_id`, `call_id`, `total_score`, `status`, `created_at`, and `flag_reason`, ordered newest-first so yesterday's calls clear before last month's.

**What puts a row in the queue.** The auto-scorer stamps `flagged_for_review = TRUE` on an evaluation whose score fell below your AI Auto-QA flag threshold. That is the flag's only trigger, which is why the queue is a morning triage list: every row is an auto-scored evaluation a human has not yet reviewed, waiting on exactly one corrective action.

**Triage workflow.** Work the queue top-down:

1. Open the flagged evaluation from the row's `evaluation_id` on **Quality → Evaluations**.
2. Rescore the criteria against the recording.
3. Submit the override — the row leaves the queue immediately.

**Clearing a flag.** The only clear action is the reviewer override on that evaluation:

```bash theme={null}
curl -X POST https://api.orbit.devotel.io/api/v1/quality/evaluations/qaeval_3d90f1c4a7/override \
  -H "Authorization: Bearer $ORBIT_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"scores": {"greeting": 20, "resolution": 45}}'
```

The override re-scores the evaluation with your per-criterion scores, re-derives the weighted total server-side, flips `flagged_for_review` to `false`, and stamps you as `reviewer_id`. The row keeps `status: "pending"` so the agent's acknowledge / appeal flow runs on the corrected score, and it keeps `auto_scored: true` as provenance — the ledger records that the AI scored first. Only an auto-scored evaluation still in `pending` is overridable; a human-graded row or an appealed one returns `409`.

**Sibling placement.** The queue shares the hub with two other supervisor rollups. `GET /quality/trends` answers "is quality moving" as a per-day average over a window, and the coaching-cards rollup answers "who do I drill." The queue answers "what needs a human now." It is also not the **Recent failures** list on the **Quality** hub — that list is the conversation-judge pipeline's triage feed for digital channels, while the queue drains the auto-QA flag on the scorecard pipeline.

The leaderboard pairing is different in kind: the board ranks, the queue drains. Read the leaderboard for recognition, then check the queue so a low rank is not just a stack of unresolved flags.

## 4. API equivalents

Card rollup, reviewer scope — compare against the form and window you care about:

```bash theme={null}
curl "https://api.orbit.devotel.io/api/v1/quality/coaching-cards?days=30" \
  -H "Authorization: Bearer $ORBIT_API_KEY"
```

```javascript theme={null}
const response = await fetch(
  "https://api.orbit.devotel.io/api/v1/quality/coaching-cards?days=30",
  { headers: { Authorization: `Bearer ${process.env.ORBIT_API_KEY}` } },
);
const { data } = await response.json();
for (const card of data.cards) {
  console.log(card.agent_id, card.avg_score, card.chips);
}
```

A card response carries the resolved window and one object per agent:

```json theme={null}
{
  "data": {
    "window": { "days": 30, "agent_id": null, "limit": 50 },
    "cards": [
      {
        "agent_id": "user_2f8Kq1XvB",
        "form_id": "qform_01",
        "evals": 12,
        "avg_score": 61.4,
        "flagged": 3,
        "flagged_share": 0.25,
        "open_plans": 1,
        "weak_criteria": [
          {
            "criterion_id": "closing",
            "label": "Closing",
            "weak_count": 4,
            "avg_percent": 55.25
          }
        ],
        "chips": [
          { "id": "qa_score", "text": "Low QA score 61.4 (below 70)" },
          {
            "id": "flag_rate",
            "text": "25% of evaluations flagged for review"
          },
          { "id": "open_plans", "text": "1 open coaching plan" }
        ]
      }
    ]
  },
  "meta": {
    "request_id": "req_01J4V9X5Z",
    "timestamp": "2026-09-18T09:00:00.000Z"
  }
}
```

The flag queue, reviewer scope:

```bash theme={null}
curl "https://api.orbit.devotel.io/api/v1/quality/compliance-flags?days=7&limit=20" \
  -H "Authorization: Bearer $ORBIT_API_KEY"
```

```javascript theme={null}
const response = await fetch(
  "https://api.orbit.devotel.io/api/v1/quality/compliance-flags?days=7&limit=20",
  { headers: { Authorization: `Bearer ${process.env.ORBIT_API_KEY}` } },
);
const { data } = await response.json();
for (const flag of data.flags) {
  console.log(flag.evaluation_id, flag.agent_id, flag.total_score);
}
```

A queue response carries the resolved window and one row per flagged evaluation:

```json theme={null}
{
  "data": {
    "window": { "days": 7, "agent_id": null, "limit": 20 },
    "flags": [
      {
        "evaluation_id": "qaeval_3d90f1c4a7",
        "agent_id": "user_2f8Kq1XvB",
        "form_id": "qform_01",
        "call_id": "call_8891",
        "total_score": 54.2,
        "status": "pending",
        "created_at": "2026-09-18T07:41:12.000Z",
        "flag_reason": "auto_score_below_threshold"
      }
    ]
  },
  "meta": {
    "request_id": "req_01J4V9X5Z",
    "timestamp": "2026-09-18T09:00:00.000Z"
  }
}
```

Reviewer-vs-agent scope applies to both reads: a role header of owner, admin, or supervisor widens `agent_id` to any agent and the queue to the whole org; a plain agent token is narrowed to the caller's own id regardless of the `agent_id` it asks for. Query fields are capped (`days` to 90, card `limit` to 50, flag `limit` to 100) so a malformed value degrades to the default instead of erroring — a strategy that keeps the morning page loadable under a typo'd bookmark.

## 5. Acknowledge and appeal, wired to cards and flags

Cards never mutate authored evaluations. A card's average re-derives on the next request when a reviewer authors a new evaluation, when an agent's appeal moves a row to `appealed` / `resolved` and out of the official aggregate, or when a reviewer override re-scores a flagged row. `flagged_for_review` clears exactly one way: the override re-stamps the row with the human reviewer and writes the flag back to `false`, and the queue drop is immediate on the next read. Nothing else — acknowledging, resolving, or re-opening a window — rewrites the ledger behind either surface.

## See also

* [Quality leaderboard](/guides/quality-leaderboard) — the ranking the coaching card sits beside
* [The call-quality evaluation lifecycle](/concepts/quality-evaluation-lifecycle) — the acknowledge / appeal flow the override feeds
* [AI Auto-QA configuration](/guides/qa-autoscore-settings) — the flag threshold that fills the queue
* [Voice coaching plans](/guides/voice-coaching-plans) — what the open-plan chips point at
* [Quality hub supervisor loop](/guides/quality-hub-supervisor-loop) — where the queue sits in the morning pass
* [Quality Management API](/api-reference/quality) — full endpoint reference