Quality leaderboard: scoring rules, badges, and thresholds

Quality → Leaderboard ranks every agent by a composed point total computed from the QA scorecards, handled calls, and CSAT responses the platform already captures; the Toplevel metric columns — QA, CSAT, and AHT — are the raw ingredients the ruleset maps into points. This guide goes behind the board: the scoring rules and badge definitions that drive it, where each counter comes from, how visibility is scoped by role, and where the model’s limits are. Along the way it covers the reading side too — comparing agents fairly and turning a rank into coaching.

1. What gamification drives in a QA program

Three artifacts make up the engagement layer:

Scores — each agent’s metrics are mapped through a point ruleset into a total, and the totals rank the board for a window. Points are a composed score, not a source metric, so the default ruleset weights QA alongside volume and satisfaction rather than letting call count alone win the board.
Badges — threshold awards evaluated per window, such as a QA average at or above a set mark, a podium finish, or a call-volume milestone. Average-based badges carry a minimum-sample guard so one lucky evaluation cannot mint them.
Thresholds — the explicit numbers in the rule and badge definitions: how many points each metric pays out, which cap a scaled rule pays up to, and which sample size a badge requires before it can earn out.

The board is navigated by window and by team: the window tabs are exactly “Today / This week / This month” — pick the period first, then narrow the roster with a queue_id filter. The rankings are window-scoped; an explicit from/to pair overrides the named window. Visibility is role-scoped, and the roles see different compositions of the same board:

Owner and admin see the full roster across every queue.
Supervisor sees only the agents in their assigned queues; an explicit queue they do not supervise is refused with a 403.
Agent sees only their own card — the anonymised rank position out of the roster, points breakdown, and earned badges — never another agent’s identity.

The agent’s own card on the leaderboard is the same My performance panel that anchors Me → My scorecard — but the leaderboard is not the self-view. The board answers “where do I rank this window?”; the scorecard answers “which of my rubrics is failing, and is it trend or blip?” Rubric pass rates, the daily trend, and top failure modes live only on the scorecard. The agent-facing read of that page is covered in Read your own QM scorecard.

2. The config behind the board — `GET /quality/gamification/config`

The definitions behind the current board are readable over the API. GET /api/v1/quality/gamification/config is reviewer-scoped (owner, admin, supervisor) and returns three things: the six numeric metric keys a rule or badge may reference (callsHandled, avgHandleSeconds, qaCount, avgQaScore, csatCount, avgCsat), the point rules, and the badge definitions. A badge may also reference the derived totalPoints or rank. Point rules come in three shapes:

Shape	Payout	Use for
`linear`	`floor(metric × pointsPerUnit)`	Volume incentives — points per handled call.
`scaled`	`round(clamp(metric / maxMetric) × maxPoints)`	Fixed-scale metrics — QA 0–100 or CSAT 1–5 mapped to a capped payout.
`belowTargetBonus`	flat `bonusPoints` when `0 < metric ≤ target`	Efficiency incentives — handle time under a target.

A badge definition holds an id, name, description, an icon the dashboard renders, a field with a comparator (gte, lte, eq) and threshold, and an optional minSamples guard naming which metric must reach a given count. The default ruleset balances volume, quality, satisfaction, and efficiency so no single behaviour dominates:

Rule	Payout
Calls handled	2 points per call (linear)
QA score	average scaled to 60 points
Customer satisfaction	average scaled to 30 points
Efficient handle time	20 bonus points when the average is 300 s or under

Default badges award a Top Performer (rank 1), a Podium Finish (top 3), QA Ace (QA average ≥ 90 with at least 3 evaluations), Centurion (100+ handled calls), and Customer Champion (CSAT ≥ 4.5 with at least 5 responses).

3. Events and counter evaluation — where the metrics come from

You do not configure event wiring; the board rolls up counters the voice and QA pipelines already write:

Calls handled and handle time — completed call records attributed to the answering agent, summed for talk seconds. avgHandleSeconds is talk seconds divided by call count.
QA score — evaluations from your quality pipeline: human-graded rows from the sampler or manual assignments, plus AI auto-score rows once a reviewer has graded them. Rows queued for review but not yet graded, and self-evaluations the agent scored themselves, are excluded from the official average so pending or self-scored zeros cannot drag it down.
CSAT — customer-satisfaction survey responses (score 1–5) tied to the agent’s handled calls.
Roster — queue membership decides which agents are in scope for the window.
Drill-through — from Quality → Agents → your-agent you can open the underlying rows: Quality → Evaluations for the graded ledger behind the QA average and Quality → Recordings for the call recordings behind the handle-time and CSAT counters. The board consumes whatever those drill paths correct.

Counts with no data in the window are zero; averages with no underlying samples stay unset and the page renders an em-dash rather than a misleading zero. The same empties feed badge evaluation, so no badge earns out on a silent period.

4. Coaching cards

Next to the ranking, each agent’s coaching card folds their official evaluations into a short drill readout — average QA score, flagged share, the weakest scorecard criteria, the count of open auto-assigned coaching plans, and badge chips (“Low QA score …”, ”…% of evaluations flagged for review”, “N open coaching plans”). Supervisors use the card to name the drill before the rank alone would; an agent opening their own card sees only their own numbers, never the league table. The chip semantics and the self-scoping rule are in Coaching cards and the compliance-flag queue.

5. Worked example — one queue, one week

Two agents on the Support Tier-1 queue for a week window, scored on the default ruleset: Agent A — 112 calls handled, 240 s average handle time, 8 evaluations averaging 92, 21 CSAT responses averaging 4.7:

Rule	Payout
Calls	112 × 2 = 224
QA	92/100 of 60 = 55
CSAT	4.7/5 of 30 = 28
Handle time	240 s ≤ 300 s → 20
Total	327

Agent A takes rank 1 and earns Top Performer, Podium Finish, Centurion (112 ≥ 100), QA Ace, and Customer Champion — first badge evaluation: the QA average of 92 clears the 90 threshold and the 8 evaluations clear the 3-sample guard, so QA Ace earns out. Agent B — 95 calls handled, 310 s average handle time, 6 evaluations averaging 88, 4 CSAT responses averaging 4.6:

Rule	Payout
Calls	95 × 2 = 190
QA	88/100 of 60 = 53
CSAT	4.6/5 of 30 = 28
Handle time	310 s > 300 s → 0
Total	271

Agent B holds rank 2 and earns Podium Finish — but second badge evaluation: Customer Champion fails despite the 4.6 average, because 4 responses undershoot the 5-sample guard. Note the points still paid out: the sample guard protects the badge only, not the score, which is why thin-sample ranks need the fairness reading in section 6. Ties sort deterministically — QA average, then calls handled, then agent id — so rank-based badges never flip on recompute.

6. Scope, roles, and the agent self-view

Over the API the same reads are POST /api/v1/quality/gamification/leaderboard (body: period, optional queue_id, optional explicit agent_ids allow-list) and GET /api/v1/quality/gamification/me for the agent-facing card. Role-scoped visibility composes like this: reviewer roles (owner, admin, supervisor) read the board; a supervisor is narrowed to their assigned queues and an explicit unsupervised queue gets a 403; a supervisor with no queue mappings sees an empty board. The agent self-view computes against the full roster with the default ruleset — the same ranking the supervisor default board shows — so an agent’s rank never disagrees with their supervisor’s view; only their own slice is returned. Reviewer-scope roles may pass agent_id to inspect another agent’s card; anyone else asking for a different agent_id gets a 403. The Scoring rules tab on the dashboard page renders the config endpoint’s defaults and lets a supervisor tune weights — the board answers rules_source: "custom" on recompute and shows a Custom weights applied chip so nobody reads a tuned preview as the standing record.

7. Read the ranking fairly — normalize for sample size

Raw rank over-credits volume, so read the columns together before concluding anything:

Points vs. Calls — a high-call agent accumulates volume points mechanically; split the rank by QA average before crediting quality.
QA average vs. coverage — the QA column averages evaluated calls, not all calls. An 88 over three evaluated calls carries more noise than an 84 over twelve; when a rank surprises you, widen the window.
CSAT vs. AHT — fast handle times that drag CSAT down are not a win; encode a floor on one metric to gate the other if your tuned ruleset rewards speed.
Badges carry sample guards; points do not — treat a top rank on a thin window as unconfirmed until the week or month view normalizes it.

A fair comparison rule of thumb: compare within the same queue, same window, and similar call counts, and weight the QA and CSAT columns over total points when coverage differs.

8. Wire the board into the QA program

Badges and ranks are program inputs, not decoration. For coach-of-the-month and unit-reporting material, pair the ranking with the monthly report — see Monthly report.

Recognition — the top of the month board, with the fairness reading above, is the recognition shortlist; badges make a good secondary filter because their sample guards screen out thin-window spikes.
Coaching targeting — a sustained bottom-decline in QA or CSAT feeds a coaching plan authored under Voice → Coaching → Coaching plans, and scores under your coaching threshold auto-assign one on the same surface — see Voice coaching plans.
Calibration — when a drill from Quality → Agents → your-agent or the evaluations ledger shows the leaderboard counting mis-graded rows, fix the reviewers or the AI flag threshold, not the agent. The board counts whatever the ledger corrected.
Competition hygiene — custom weights are a preview for evaluating a scheme, never a mid-competition edit. Pin one ruleset per period and change it between periods; re-ranking mid-period rewrites history.

Operate the board without poisoning the floor — Never shame the bottom: read bottom ranks in private, follow the trend, not the absolute rank, and watch for agents optimizing the measured columns at the expense of unmeasured ones — that is a rules problem, not an agent problem.

9. Limits

Badges and points are window-scoped. Every definition reflects the current period only — the board recomputes when you open it and keeps no historical badge ledger. An explicit from/to pair overrides the named day / week / month windows; month means a trailing 30 days.
Team scope is a filter, not a saved config. queue_id (or an explicit agent_ids list) narrows the roster per request; nothing persists the narrowed competition.
Custom rules are per-request. Tuned point_rules / badge_definitions sent to the endpoint preview a scheme and report rules_source: "custom"; the default ruleset is what the standing board and the agent self-view run on.
No persistence, no write surface. The layer is a read-time rollup over existing QA, call, and CSAT aggregates — there is nothing to backfill and nothing to reset when you change your evaluation form; the next window recomputes from the data your pipeline produces.

10. API parity — request and response samples

The two reads behind the board are public API surfaces you can drive from a script or a reporting pipeline: POST /api/v1/quality/gamification/leaderboard for the ranked board, and GET /api/v1/quality/gamification/me for the agent self-view card. The leaderboard request body takes a named period (day, week, or month; defaults to week) and an optional queue_id to narrow the roster to one queue. An explicit ISO from/to pair overrides the named window, and an optional agent_ids allow-list pins the roster to named agents:

Field	Type	Meaning
`period`	string	Named window: `day` (24 h), `week` (7 days), or `month` (trailing 30 days). Defaults to `week`.
`queue_id`	string (optional)	Restrict the roster to one queue.
`from` / `to`	ISO 8601 (optional)	Explicit window override — both must be sent to take effect.
`agent_ids`	string[] (optional, max 500)	Restrict the roster to an explicit agent allow-list.

Access is role-scoped, and the role decides which read you may call:

Role	Leaderboard	Self-view (`/gamification/me`)
Owner / admin	Full roster, any queue.	Own card; may pass `agent_id` to inspect another agent.
Supervisor	Limited to assigned queues — an explicit `queue_id` the supervisor does not supervise is rejected with `403 FORBIDDEN`.	Own card; may pass `agent_id`.
Agent	`403 FORBIDDEN` — reviewer-scope only.	Own card only; passing another agent’s `agent_id` is rejected with `403 FORBIDDEN`.

A supervisor with no queue mappings sees an empty board rather than an error. Read the board for a queue over the current week:

curl -X POST https://api.orbit.devotel.io/api/v1/quality/gamification/leaderboard \
  -H "Authorization: Bearer $ORBIT_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"period": "week", "queue_id": "queue_support_l1"}'

The same read from Node.js:

const response = await fetch(
  "https://api.orbit.devotel.io/api/v1/quality/gamification/leaderboard",
  {
    method: "POST",
    headers: {
      Authorization: `Bearer ${process.env.ORBIT_API_KEY}`,
      "Content-Type": "application/json",
    },
    body: JSON.stringify({ period: "week", queue_id: "queue_support_l1" }),
  },
);
const { data } = await response.json();
for (const entry of data.entries) {
  console.log(entry.rank, entry.agentName, entry.totalPoints);
}

Read your own card — any authenticated agent scope:

curl "https://api.orbit.devotel.io/api/v1/quality/gamification/me?period=week" \
  -H "Authorization: Bearer $ORBIT_API_KEY"

The same read from Node.js:

const response = await fetch(
  "https://api.orbit.devotel.io/api/v1/quality/gamification/me?period=week",
  {
    headers: {
      Authorization: `Bearer ${process.env.ORBIT_API_KEY}`,
    },
  },
);
const { data } = await response.json();
console.log(data.me.rank, data.me.totalPoints, data.me.badges);

A complete leaderboard response carries the resolved window, the scope echo, the ruleset the board was computed with, and per-agent entries ordered by rank — plus the standard meta envelope:

{
  "data": {
    "window": {
      "from": "2026-09-11T09:00:00.000Z",
      "to": "2026-09-18T09:00:00.000Z",
      "period": "week"
    },
    "scope": { "queue_id": "queue_support_l1", "agent_ids": null },
    "rules_source": "default",
    "entries": [
      {
        "rank": 1,
        "agentId": "user_2mN4zR7Ld",
        "agentName": "Dana Okafor",
        "totalPoints": 337,
        "breakdown": [
          { "ruleId": "calls-volume", "label": "Calls handled", "points": 236 },
          { "ruleId": "qa-quality", "label": "QA score", "points": 55 },
          {
            "ruleId": "csat-satisfaction",
            "label": "Customer satisfaction",
            "points": 26
          },
          {
            "ruleId": "aht-efficiency",
            "label": "Efficient handle time",
            "points": 20
          }
        ],
        "badges": [
          {
            "id": "top-performer",
            "name": "Top Performer",
            "description": "Ranked #1 on the leaderboard for this period.",
            "icon": "trophy"
          }
        ],
        "metrics": {
          "callsHandled": 118,
          "avgHandleSeconds": 264,
          "qaCount": 7,
          "avgQaScore": 92.14,
          "csatCount": 31,
          "avgCsat": 4.4
        }
      }
    ]
  },
  "meta": { "request_id": "req_01J4V9X5Z", "timestamp": "2026-09-18T09:00:00.000Z" }
}

The self-view response mirrors the same envelope with data.me instead of entries: your rank on the full org board, totalAgents, the per-rule breakdown, earned badges, and the raw metrics. Averages render null when you have no sample; hasActivity: false marks a silent window so you can render an empty state rather than a zero score.

WhatsApp

Developer console

Contact center

Guides

Messaging Controls

Audience & Support Tools

Quality leaderboard: scoring rules, badges, and thresholds

Quality leaderboard: scoring rules, badges, and thresholds

1. What gamification drives in a QA program

2. The config behind the board — `GET /quality/gamification/config`

3. Events and counter evaluation — where the metrics come from

4. Coaching cards

5. Worked example — one queue, one week

6. Scope, roles, and the agent self-view

7. Read the ranking fairly — normalize for sample size

8. Wire the board into the QA program

9. Limits

10. API parity — request and response samples

See also

​Quality leaderboard: scoring rules, badges, and thresholds

​1. What gamification drives in a QA program

​2. The config behind the board — GET /quality/gamification/config

​3. Events and counter evaluation — where the metrics come from

​4. Coaching cards

​5. Worked example — one queue, one week

​6. Scope, roles, and the agent self-view

​7. Read the ranking fairly — normalize for sample size

​8. Wire the board into the QA program

​9. Limits

​10. API parity — request and response samples

​See also

Quality leaderboard: scoring rules, badges, and thresholds

1. What gamification drives in a QA program

2. The config behind the board — `GET /quality/gamification/config`

3. Events and counter evaluation — where the metrics come from

4. Coaching cards

5. Worked example — one queue, one week

6. Scope, roles, and the agent self-view

7. Read the ranking fairly — normalize for sample size

8. Wire the board into the QA program

9. Limits

10. API parity — request and response samples

See also