Skip to main content

​
Per-agent coaching cards and the compliance-flag queue

Two read surfaces on Quality answer the supervisor’s morning question — “who needs coaching, and which low auto-scores do I clear first” — before the leaderboard or trend pages are even opened:
  • Coaching cards — a per-agent rollup over reviewer-authored evaluations: each agent’s average QA score, the share of their evaluations flagged for review, their weakest scorecard criteria, the count of open auto-assigned coaching plans, and short badge chips. Cards are sorted lowest-average-first so the drilling list orders itself.
  • The compliance-flag queue — a newest-first list of evaluations whose flagged_for_review bit is set, so the auto-scorer’s below-threshold signals clear in a fixed order. Each row carries flag_reason: "auto_score_below_threshold" — the one reason a flag can exist today.
Both are read-only rollups over the evaluation ledger: they never mutate a scored evaluation, and only a human override changes what they report.

​
1. Who reads which card

Access splits into reviewer scope and agent self-scope, and each endpoint applies the split identically: The self-scope narrowing is enforced at request time — an agent passing another agent’s agent_id still gets their own card back, never a teammate’s. The coaching card is also scoped to what it reports: it contains one agent’s own numbers and chips, never the league table. An agent in the self-view never sees a rank or another agent’s identity.

​
2. The coaching card

For a reviewer, days (1–90, default 30), agent_id, and limit (default 50, clamped at 50) narrow the rollup. A malformed value falls back to the default instead of erroring, so a dropped query parameter degrades rather than blocks the page. Each returned card carries:
  • agent_id and form_id — the agent and the scorecard form the rollup ran on.
  • evals — the evaluation count behind the averages.
  • avg_score — mean QA score over those evaluations, null on an empty window.
  • flagged and flagged_share — how many evaluations were flagged for review, and that count as a share of evals.
  • open_plans — coaching plans still active for this agent.
  • weak_criteria — up to five weakest scorecard criteria, rolled up per agent with the criterion label from the form definition. A criterion counts as weak when its normalised percent falls below 70, ranked by most-often-weak then lowest average.
  • chips — the badge chips that make the card scannable.
The chip semantics are fixed — three chip types, each readable straight off the card: In the dashboard these same chips render on Quality → Leaderboard as the per-agent coaching card next to the board, so a rank and the drilling context sit on one screen. A chip fires because a threshold tripped, so it is a cue to investigate the cut behind it — a flag_rate chip on two evaluations reads differently than the same chip on twenty. Both endpoints only count official evaluations. Excluded before any average: an agent’s self-evaluation, un-scored auto-sample and CSAT-trigger placeholder rows, and evaluations that moved into appealed or resolved (a superseded score drops out of the denominator). A thin window of two graded calls produces the same shape as a dense one — read the chips against evals before acting on them.

​
3. The compliance-flag queue

GET /quality/compliance-flags takes the same days (default 30), agent_id, and limit (default 50, clamped at 100) parameters. Each row carries evaluation_id, agent_id, form_id, call_id, total_score, status, created_at, and flag_reason, ordered newest-first so yesterday’s calls clear before last month’s. What puts a row in the queue. The auto-scorer stamps flagged_for_review = TRUE on an evaluation whose score fell below your AI Auto-QA flag threshold. That is the flag’s only trigger, which is why the queue is a morning triage list: every row is an auto-scored evaluation a human has not yet reviewed, waiting on exactly one corrective action. Triage workflow. Work the queue top-down:
  1. Open the flagged evaluation from the row’s evaluation_id on Quality → Evaluations.
  2. Rescore the criteria against the recording.
  3. Submit the override — the row leaves the queue immediately.
Clearing a flag. The only clear action is the reviewer override on that evaluation:
The override re-scores the evaluation with your per-criterion scores, re-derives the weighted total server-side, flips flagged_for_review to false, and stamps you as reviewer_id. The row keeps status: "pending" so the agent’s acknowledge / appeal flow runs on the corrected score, and it keeps auto_scored: true as provenance — the ledger records that the AI scored first. Only an auto-scored evaluation still in pending is overridable; a human-graded row or an appealed one returns 409. Sibling placement. The queue shares the hub with two other supervisor rollups. GET /quality/trends answers “is quality moving” as a per-day average over a window, and the coaching-cards rollup answers “who do I drill.” The queue answers “what needs a human now.” It is also not the Recent failures list on the Quality hub — that list is the conversation-judge pipeline’s triage feed for digital channels, while the queue drains the auto-QA flag on the scorecard pipeline. The leaderboard pairing is different in kind: the board ranks, the queue drains. Read the leaderboard for recognition, then check the queue so a low rank is not just a stack of unresolved flags.

​
4. API equivalents

Card rollup, reviewer scope — compare against the form and window you care about:
A card response carries the resolved window and one object per agent:
The flag queue, reviewer scope:
A queue response carries the resolved window and one row per flagged evaluation:
Reviewer-vs-agent scope applies to both reads: a role header of owner, admin, or supervisor widens agent_id to any agent and the queue to the whole org; a plain agent token is narrowed to the caller’s own id regardless of the agent_id it asks for. Query fields are capped (days to 90, card limit to 50, flag limit to 100) so a malformed value degrades to the default instead of erroring — a strategy that keeps the morning page loadable under a typo’d bookmark.

​
5. Acknowledge and appeal, wired to cards and flags

Cards never mutate authored evaluations. A card’s average re-derives on the next request when a reviewer authors a new evaluation, when an agent’s appeal moves a row to appealed / resolved and out of the official aggregate, or when a reviewer override re-scores a flagged row. flagged_for_review clears exactly one way: the override re-stamps the row with the human reviewer and writes the flag back to false, and the queue drop is immediate on the next read. Nothing else — acknowledging, resolving, or re-opening a window — rewrites the ledger behind either surface.

​
See also