Skip to main content

​
Answer Disposition Bot: post-call labels, corrections, and QA routing

The Answer Disposition Bot is the auto-labelling lane that assigns one outcome label (Resolved, Refund issued, Escalated, Wrong number) to every closed conversation that an agent didn’t stamp by hand. It keeps outcome coverage at 100% — instead of reporting on the 40% of closes an agent happened to label — while every label it writes stays correctable by the human it concerns. This guide covers the operator workflow: when the bot fires, the taxonomy it writes against, how agents and QA reviewers correct it, where its labels feed scoring, and what to check when it goes quiet. For the taxonomy-design side (how to structure the label sets the bot picks from), see Design inbox dispositions; for the per-queue-wrap-up catalog, see Wrap-up codes; for tenant-wide, many-per-call tags, see Call disposition tags. Dashboard surfaces: Inbox → Settings → Dispositions (label sets and distribution), Quality → Evaluations (QA pipeline). Authentication: Clerk session (Authorization: Bearer <token>) or API key (X-API-Key).

​
1. When the bot runs — post-call, never mid-conversation

The bot fires after a conversation closes, never while the thread is live. There is no live-assist behaviour here — live guidance to the agent is a separate surface (whisper coaching); the disposition bot’s only decision is “what did this conversation end with”. The firing sequence on a close event:
  1. The close completes. An agent had the option to stamp a label from the close picker. If they stamped one, the bot never runs — manual labels always win.
  2. If the close went through unstamped, a background worker (not the request path, so the close can’t hang on an LLM call) loads the last 50 messages of the thread plus the auto-generated summary if one exists, redacts phone numbers, email addresses, and card-shaped identifiers from both, and asks a fast classifier to pick one label from your tenant’s default label set. The bot returns the label, a confidence between 0 and 1, and one sentence of reasoning.
  3. The label row is written on the conversation with source = ai_auto, the one-line rationale, and the confidence. A conversation.disposed webhook event fires for any downstream consumers.
  4. A close with too little content to classify (fewer than two message bodies) or no default label set configured is skipped silently — the close still succeeds; the distribution panel just shows a null-outcome bucket until your sets are in place.
Re-opening then re-closing a conversation replaces the bot’s prior assignment — the label row is upserted keyed on the conversation, so re-runs converge onto one label instead of layering. Voice calls: the same bot fires for voice conversations closed through the inbox channel. It is distinct from the two agent-confirmed voice lanes — the queue-owned wrap-up codes an agent must pick before going available again when the queue gates on requireDisposition, and the call disposition tags a caller stamps in the dashboard. On the voice softphone an agent can also fetch a suggestion of which wrap-up code to pick (a predictive hint read against the queue’s own catalog); that read is advisory only — nothing is written until the agent confirms.

​
Threshold checks the bot applies

The bot gates on three thresholds before it writes anything:
  • Minimum content — at least two message bodies; otherwise it skips rather than guessing.
  • Default label set exists — if your tenant has no set marked default for the conversation’s type, the bot skips (it never picks from a non-default set on its own).
  • Confidence floor — when the classifier’s confidence lands under the review threshold (0.6 platform default), the label is still written but flagged needs_review, so a borderline guess enters the human-review lane instead of passing as authoritative.
Each guard exists to bound blast radius: the bot errs toward leaving a conversation unlabeled rather than stamping a wrong label at scale.

​
2. The taxonomy it writes — outcome labels, not wrap-up codes

The bot picks one label from one label set per conversation, where a label set is a named list of up to 50 outcome labels scoped to a conversation type (inbox, voice, or any) and one per type is marked default. Manage sets under Inbox → Settings → Dispositions → Label sets (owner/admin only); each label carries a display text (mutable), a slug id (immutable once assigned — renames follow stamps), an optional colour, and an optional description. The description is the bot’s guidance channel — “when should it pick this label” — so sets whose descriptions overlap are the root of most high override rates. Flat sets classify cleaner than trees. A two-level hierarchy like Support → Billing → Refund gives the classifier three candidates it can’t tell apart, which inflates the override rate until the set is flattened; keep each label describable in one sentence. Three axes, kept separate on purpose: Use label sets when you want the outcome axis; reach for wrap-up codes when a queue needs to gate an agent’s release; tags when you need overlapping, many-per-call slicing.

​
3. Appealing and correcting the bot’s labels

Every bot-written label is correctable by the human it concerns — this is the tenant-owned control plane, and no Devotel support ticket is needed for any of it.

​
An agent re-stamps their own conversation

The assigned agent can re-stamp the closed conversation with the correct label. The duplicate-label write replaces the bot’s assignment and flips the row’s source to agent_override, so dashboards count the correction as a human-over-AI delta rather than a fresh manual label. Use it when the agent knows the bot got the outcome wrong on their own thread. One line on who may correct what: voice wrap-up codes are protected by an owning-agent gate — only the agent who handled the call records its outcome, with owner/admin and platform staff keeping a blanket capability for after-the-fact corrections (cross-agent stuffing rejected). The same intent applies to label sets: the assigned agent corrects their own thread; a supervisor or QA reviewer can correct anything. An agent cannot restamp somebody else’s conversation.

​
QA reviewers and the appeal workflow

For a bot label that feeds a QA evaluation, the appeal loop is the canonical fix path:
  1. The evaluated agent appeals the score under Quality → Evaluations, naming the wrong disposition in the appeal note (the acknowledge/appeal/resolve lifecycle is on QA Evaluations).
  2. The reviewer resolves the appeal and re-uses the re-stamp path to set the corrected label. The row flips to agent_override and the appeal note documents why — an exportable audit trail, not a shadow edit.
  3. Supervisor overrides run through the same path; every override writes to the audit log with the actor attribution.

​
Team-level appeal loops

Two other hooks close the loop:
  • Coaching queue. The QM coaching loop surfaces low-scoring conversations for a supervisor with per-conversation notes. A wrong-outcome label that skews the score is exactly the kind of correction that queue exists to catch.
  • Statistics endpoint. GET /api/v1/inbox/dispositions/stats reports per-label counts, average bot confidence, and — the number that matters — the override rate per label: the fraction of bot assignments humans later corrected. A label whose override rate stays high is a taxonomy problem, not a coaching problem; tighten its description or split it before you re-train agents on it.

​
4. Where the bot’s output feeds QA scorecards

Disposition labels annotate the same concluded conversation a QA reviewer grades:
  • Quality → Evaluations. Reviewers (or the AI auto-scorer when enabled) grade a conversation against your weighted form; the disposition label annotates the outcome axis a reviewer sees beside the score. Auto-scored evaluations carry auto_scored = true provenance so you can tell a bot-graded score from a reviewer one; details are on the QA Evaluations guide and the call-rubric QA scorecards page.
  • Distribution panel. The Inbox → Settings → Dispositions → Distribution tab aggregates labels over the trailing 30 days. Agents and supervisors read outcome mix here; QA liaises on the numbers before arguing about per-criterion scores.
  • Fix path for a wrong label feeding a score. Appeal the evaluation in Quality → Evaluations, resolve it, and re-stamp the correct label as agent_override. The appeal note travels with the corrected label — the audit chain a QA export expects.
The check above is how a single bot label moves through a scorecard: the label annotates; the form weights grade; the appeal loop corrects either side.

​
5. Troubleshooting — the bot stops, misfires, or skips

Diagnose the bot at five levels, cheapest first:
  1. Label-set problem. Inbox → Settings → Dispositions → Label sets — confirm a set is marked default for the conversation type. No default means the bot skips silently and your close flow never labels.
  2. Hook problem, post-call. The bot hooks the conversation-close event. If closes work but nothing fires, check the close path can reach the bot’s worker (event loop) — a stuck webhook queue is the usual culprit.
  3. Voice-gateway hook. For a voice conversation, the label depends on the transcript being persisted. A enabled voice lane requires the gateway’s recording → transcript chain to complete; the bot can’t label a voice call that never left a transcript. If the call closed and no transcript landed, the transcript chain is where to look — not the bot’s judge.
  4. Budget and caps. The bot’s LLM calls spend against the tenant wallet. Those calls can be throttled or cut entirely when the org’s spend caps trip pause_outbound or block_outbound, or when the per-tenant LLM cap cache hits. A bot that went quiet at the same moment a billing alert fired is a caps problem, not a classifier problem.
  5. Canvas/flows check. When a conversation or a call flows through the Agent Studio canvas, confirm the conclusion path persists to the inbox close event the bot hooks. A canvas step that bypasses the close path bypasses the label.
Other recurring causes, easiest first: the taxonomy is_active flag flipped off; the label description widened so the classifier’s candidate overlap grew; re-close retries stacking duplicates until the upsert converges (reads return the latest row). Work the checklist in order. A majority of silent-bot reports resolve at step 1 (a tenant that has genuinely never marked a default set).