​

QA scorecard trends: reading the org-wide score series

​

1. What the trend answers — and what counts as official

• Self-evaluations never enter the trend. An agent scoring their own conversation is excluded (agent equals reviewer means it is not an official score) — the same rule the leaderboard applies, so the trend and the ranking can never drift on self-grading.
• Appealed and resolved rows are out. An appealed score is under dispute and a resolved row is superseded; both are audit trail, not signal. Pending and acknowledged rows do count — acknowledgement is a notification step, the score stands either way.
• The day bucket is the evaluation’s creation day. A backfilled score lands in the day it was authored, not the day the call happened, so heavy backfill weeks show up as eval-volume spikes on recent days.

​

2. GET /quality/trends walk-through

GET /api/v1/quality/trends?days=30&agent_id=agent_123&form_id=form_456

​

Query parameters

​

Response shape

{
  "data": {
    "window": { "days": 30, "form_id": null, "agent_id": null },
    "totals": {
      "total_evals": 412,
      "avg_score": 78.4,
      "scored_agents": 23,
      "active_forms": 2
    },
    "trend": [
      { "day": "2026-09-01", "evals": 14, "avg_score": 76.2 },
      { "day": "2026-09-02", "evals": 12, "avg_score": 79.8 }
    ],
    "by_agent": [
      { "agent_id": "agent_123", "evals": 31, "avg_score": 84.1 },
      { "agent_id": "agent_087", "evals": 28, "avg_score": 61.3 }
    ],
    "by_form": [
      { "form_id": "form_456", "evals": 240, "avg_score": 79.9 },
      { "form_id": "form_789", "evals": 172, "avg_score": 76.0 }
    ]
  },
  "meta": { "request_id": "req_...", "timestamp": "2026-09-15T08:00:00.000Z" }
}

• window echoes the resolved scope — what was actually applied after clamping, not necessarily what you sent.
• totals is the window rollup: official evaluations scored, mean weighted total (or null when nothing was scored), distinct scored agents, and distinct scorecard forms in play. Read avg_score as the org’s QA level for the window and evals as how much evidence backs it.
• trend is the per-day time series, chronological, one point per day that had evaluations: the day’s evaluation count and its average score. Days with zero evaluations produce no point — treat a missing day as “nothing scored”, not “scored zero”.
• by_agent ranks up to 25 agents by evaluation volume (heaviest first) with each agent’s average — the org-level “compare agents” cut.
• by_form is the same cut per scorecard form, so you can tell whether two rubrics grade the same team consistently.

​

Example request and a minimal dashboard render

curl "https://api.orbit.devotel.io/api/v1/quality/trends?days=30&agent_id=agent_087" \
  -H "X-API-Key: $ORBIT_API_KEY"

import { Orbit } from '@devotel-orbit/node';

interface TrendPoint { day: string; evals: number; avg_score: number | null; }
interface TrendsData {
  window: { days: number; form_id: string | null; agent_id: string | null };
  totals: { total_evals: number; avg_score: number | null; scored_agents: number; active_forms: number };
  trend: TrendPoint[];
  by_agent: { agent_id: string; evals: number; avg_score: number | null }[];
  by_form: { form_id: string; evals: number; avg_score: number | null }[];
}

const orbit = new Orbit({ apiKey: process.env.ORBIT_API_KEY! });
const { data } = await orbit.request<TrendsData>(
  'GET',
  '/api/v1/quality/trends?days=30&agent_id=agent_087',
);

function scoreColor(avg: number | null): string {
  if (avg === null) return 'gray';
  if (avg >= 80) return 'green';
  if (avg >= 60) return 'amber';
  return 'red';
}

for (const point of data.trend) {
  renderPoint({ day: point.day, value: point.avg_score, color: scoreColor(point.avg_score) });
}

​

3. Supervisor workflows

​

Spot drift after coaching, not after a quarter

1. Before the coaching session, note the agent’s trend over the last 30 days (?agent_id=...) and the rubric group that is failing (the per-agent scorecard’s group breakdown names it).
2. Run the coaching — a calibration disagreement, a one-on-one, or a Practice Studio scenario. Record the date.
3. Re-read the same agent-scoped trend a week later. A working intervention walks the daily average up off its floor; a flat series on unchanged evaluation volume means the coaching missed the cause, not that the agent ignored it.

​

Compare agents on evidence, not on rank

• High volume, low average is a coaching target: row evals shows the score is well-evidenced, and the trend drills it by day to confirm it is a pattern, not a bad Monday.
• Low volume, any average is a sampling problem first: an agent with two evaluations in 30 days has no trend — fix the sampling quota before drawing a conclusion from the number.

​

Tie trend dips to practice sessions

1. Read the dip. The org trend or an agent-drilled trend drops over several days. Open the per-agent scorecard from the hub roster to find the failing rubric group (“closing”, “identity verification”) before naming any scenario — sending the agent to practice first is guessing (see QA Evaluations).
2. Diagnose against the evaluation. Pull the actual evaluations behind the dip from the ledger and read the lowest-scoring criteria; the worked conversation beats the average.
3. Rehearse it. Author a Practice Studio scenario for the failing pattern and assign it. The agent’s sessions are scored automatically, but they do not re-enter the evaluation ledger — rehearsal volume never cancels a bad official score.
4. Confirm on the trend. The next real evaluations on the failing rubric group move the agent’s trend; if a week of normal scoring volume shows no recovery, the scenario targeted the wrong skill.

​

4. Reading pitfalls

• Do not trend before the evaluators have calibrated. If reviewers score the same conversation apart, the daily average is reviewer noise, not quality movement; run a calibration session first (see Build a contact-center QA program).
• Backfill weeks inflate the right edge. A reviewer catching up on a backlog lands many evaluations on the same day — the evals count on that point tells you it is a volume artifact.
• An empty window is not a failing window. totals.avg_score: null with total_evals: 0 means nothing official was scored; the trend has no points, not low points.

​

See also

• QA Evaluations — the scorecard pipeline the trend is computed over
• Quality leaderboard — the composed ranking that pairs with the QA-only trend
• The Quality hub: run the supervisor loop — the weekly loop across all four QM surfaces
• Train agents with AI roleplay in Practice Studio — the rehearsal step a trend dip assigns
• Build a contact-center QA program — calibration and evaluator governance behind the official scores