Skip to main content

​
KPI alerts on business metrics

Define org-scoped alert rules over the cross-pillar KPIs your ops team owns: CSAT, NPS, CES, LLM spend, and AI containment. When the metric breaks your threshold — or deviates sharply from its own learned baseline — the rule fires an in-app notification with a deep link straight to the metric page. This is the business-KPI counterpart to Usage & delivery anomaly alert rules, which covers SMS delivery rate, outbound volume, and spend. Use both: usage alerts watch the transport; KPI alerts watch the customer experience and AI cost. Rules are read-only over your survey and agent-conversation data. The only writes are alert-rule state and the notification it fires — a rule never sends outbound traffic.

​
When business-KPI alerting matters

  • CSAT / NPS / CES drops — satisfaction moves silently; by the time someone reads the survey report, a broken flow or a bad agent has eroded scores for days. A threshold rule on the trailing window catches the slide the same day.
  • LLM-spend ceilings — AI agent costs scale with usage, and a retry loop or a runaway prompt can multiply spend in hours. Cap the trailing total and act before the budget conversation.
  • AI containment dropoffs — when containment slips, humans pick up conversations the agent used to resolve. That’s staffing pressure you can see in this number before you feel it in the queue.
For every metric, decide between two trigger styles: a fixed threshold (“tell me when CSAT falls below 80%”) or an anomaly detector (“tell me when CSAT dives relative to its own recent norm”). The rest of this page explains how each evaluates.

​
The two alert types

​
Threshold mode (default for any rule)

Aggregates the metric over a trailing window of window_days (1–90, default 7) and compares it to your fixed threshold with your comparator:
  • lt / lte — “must stay above X” (e.g. CSAT below 80%, containment below 70%).
  • gt / gte — “must stay below X” (e.g. LLM spend above $250/day window).
NPS is evaluated in points, CSAT and containment in percent, CES on its score scale, and LLM spend in USD — thresholds are entered on the metric’s native unit.

​
Anomaly mode

Compares the newest complete UTC day against a learned 28-day baseline using a z-score classifier. You set no threshold: “unusual for your account” is the trigger, just like the usage-side anomaly rules. For CSAT/NPS/CES/containment it flags a drop; for LLM spend, a spike. Days with no data are excluded from the baseline rather than zero-filled, and a data-less current day reports insufficient_data instead of firing a false alarm. Pick threshold when there’s a contractual or budgeted number (SLA CSAT, spend cap). Pick anomaly when you don’t know the right number yet and want the metric to tell you when it moves unusually.

​
Create a rule in the dashboard

  1. Open Insights → Alerts in the dashboard (or go directly to /insights/alerts).
  2. Click Create rule.
  3. Name the rule (e.g. “Weekly CSAT below 80%”).
  4. Pick the Metric — CSAT, NPS, CES, LLM spend, or AI containment.
  5. Pick the Trigger — Threshold or Anomaly.
  6. For threshold rules, set the Condition comparator and the numeric Threshold. Anomaly rules skip this — the baseline replaces it.
  7. Set the Window (days) the threshold aggregates over (anomaly rules always compare daily against the 28-day baseline; the field is carried but the evaluation is daily).
  8. Toggle Enabled and save.
The rules table shows each rule’s current value, last status, and an Enabled toggle; the Evaluate now button runs the same evaluation the scheduler runs, on demand. Below the table, the Fired alerts feed lists the last 50 notifications with a View metric link into the breached surface.

​
Endpoints

All endpoints are tenant-scoped under the analytics surface, use the standard { "data": ..., "meta": ... } envelope, and persist rules on your organization’s settings. A tenant can hold up to 50 rules. A scheduler sweep also evaluates every opted-in tenant continuously, so rules fire without you calling evaluate.

​
Auth requirements

  • Reads (GET ..., GET .../events) — a valid API key/JWT whose scopes include analytics:read.
  • Writes and evaluate-now (POST, PATCH, DELETE, POST .../evaluate) — additionally require an org role of owner, admin, or developer. Create, update, and delete are recorded in the audit log with the acting user.

​
Metric reference

Satisfaction metrics are computed from your survey responses with the same scoring the Surveys dashboard uses (NPS promoters minus detractors, CSAT/CES response averages); LLM spend and containment aggregate your agent conversations the same way the LLM-spend and Containment pages do — so an alert fires on exactly the numbers the dashboard shows.

​
Field reference (create / update body)

notify_channels currently accepts only in_app. Alerts surface through the dashboard notification channel (the bell / Notification Center). To integrate elsewhere, poll GET /analytics/kpi-alert-rules/events.

​
Notification lifecycle

Every evaluation tick updates each enabled rule’s runtime fields (last_evaluated_at, last_status, last_value) — even in a breach. A fired-alert event is appended (feed capped at 50, newest first) and a notification pushed only when:
  • the rule transitions into breached, or
  • it stays in breach and the cooldown_hours re-notify window (default 24h) has elapsed.
The notification lands in the bell dropdown and the Notification Center with severity warning, titled KPI alert: <your rule name>, the breach sentence as the body, and a deep link to the metric surface (/insights/surveys, /insights/llm-spend, or /insights/containment). See In-dashboard notifications and the Notification Center for triage semantics.

​
Event payload

​
Worked examples

​
CSAT threshold — guard a customer-experience floor

​
NPS anomaly — catch a sudden promoter slide

No comparator or threshold: the rule learns the account’s own baseline.

​
LLM-spend cap — guard the AI budget

​
Containment anomaly — catch a handoff regression

​
Update, evaluate, and read the feed

Update any field without recreating the rule:
Evaluate now (optional — the scheduler evaluates on its own cadence):
Read the fired feed:

​
Troubleshooting

422 says “comparator is required for a threshold rule”. Threshold rules must carry both a comparator and a finite threshold at create time, and updates can’t leave a threshold rule without either. Conversely, anomaly rules must not be modeled with a comparator — omit comparator/threshold entirely when mode: "anomaly"; the rule derives its trigger from the learned baseline. This mirrors the convention in Usage & delivery anomaly alert rules: threshold mode requires comparator/threshold, anomaly mode rejects them. A rule’s last_status reads insufficient_data. The metric could not be resolved in the window — usually zero survey responses (or zero agent conversations, for LLM spend / containment) in the window. For anomaly rules, fewer than five baseline days with data also reports insufficient_data. Evaluation reports no value rather than firing on a fabricated number, so give a new tenant some traffic before expecting anomaly coverage. The rule fires fewer notifications than expected. Cooldown semantics: while a rule stays in breach, it re-notifies only once per cooldown_hours (default 24h). New breach transitions always notify immediately. You want email or webhook instead of the dashboard notification. The only configured channel today is the dashboard notification (notify_channels: ["in_app"]). Poll the fired-event feed to integrate alerts into your own systems. The anomaly rule fires on a metric you consider normal. The detector flags deviation from your own recent baseline, not an absolute range. If a new baseline is genuinely your normal, the rule goes quiet once the 28-day window absorbs the shift; if you actually want a fixed boundary, switch to threshold mode.