> ## Documentation Index
> Fetch the complete documentation index at: https://docs.orbit.devotel.io/llms.txt
> Use this file to discover all available pages before exploring further.

# Score AI-agent quality per language

> Read containment, resolution, escalation, sentiment, and CSAT sliced by the detected conversation language — then act on the locale where your agent underperforms.

# Language quality

A blended containment number can hide a weak locale. An agent that resolves 80% of English conversations but escalates every German one still shows a healthy org-wide average — and the gap stays invisible until customers in that language churn. The **Per-language AI quality** dashboard slices the AI-agent quality signals Orbit already records by the language each conversation was held in, so you can spot the underperforming locale and fix it.

The dashboard lives at **Insights → Per-language AI quality** (`/insights/language-quality`). The same data is available over the API at `GET /api/v1/insights/language-quality`. This guide explains what the per-language metrics mean, how Orbit decides which language a conversation was in, how to read the page, and what to change when one language drags.

## What per-language quality metrics are

Every metric on the dashboard is one you already know from the [AI containment](/guides/insights-dashboards#ai-containment--resolution--insights--containment) and sentiment surfaces — recomputed within one language bucket:

* **Containment rate** — the share of conversations the AI agent closed without a human handoff. The inverse of escalations.
* **Escalation rate** — the share handed off to a human. A high escalation rate in one language means the agent gives up (or the handoff rules fire) more often in that locale.
* **Resolution rate** — the share that actually resolved, whether the AI or a human closed it. Resolution is the live-traffic analogue of a passed eval: the conversation reached a resolved status or passed an outcome rubric.
* **Sentiment** — the average of the signed conversation-sentiment score, over the conversations that carried one.
* **CSAT** — the average of the end-of-conversation satisfaction rating, over the conversations where a rating was captured.

Rates are percentages (0–100) with one decimal; score averages are means over the conversations that carried a score. When a slice has no conversations in the window — or none that carried the score — the dashboard shows "—" instead of a fabricated `0%`. The totals card across the top of the page is the org-wide rollup, so you can compare each language against the blended number it hides behind.

## How language detection works

Each conversation is assigned to one language bucket:

1. **Persisted tag first.** When the conversation carries a stored language tag (the same tag multi-locale inbox macros rely on), that tag is authoritative. Locale forms like `en-US` normalise to the ISO 639-1 code `en`; a tag the detector does not know is ignored in favour of detection.
2. **Detection otherwise.** For un-tagged conversations, Orbit runs its bundled language detector over the first substantial customer message and buckets the conversation under the dominant language. This is the same detector the agent runtime uses, so today's un-tagged history still segments — and once a persisted tag appears it takes over transparently.

The detector emits a fixed set of ISO 639-1 codes (English, Arabic, Hebrew, Chinese, Japanese, Korean, Russian, Ukrainian, Greek, Hindi, Thai, French, Spanish, German, Italian, Portuguese, Dutch, and Turkish). Conversations with no usable signal land in an `unknown` bucket rather than being guessed at.

Two honesty rules to know when reading the numbers:

* **Detection is per conversation, so the scan is bounded.** Orbit reads one lightweight row per conversation (the pre-computed sentiment/CSAT scalars plus a short text sample) instead of a pure aggregate. When your window holds more conversations than the scan cap (20,000), the page labels itself as covering the *most recent* conversations and shows a truncation note — shrink the window to cover the full range.
* **Totals are never truncated.** The rollup at the top is summed across every language before the displayed rows are capped, so the headline numbers are always the full window.

## Read the per-language dashboard

Open **Insights → Per-language AI quality**. Pick a window (24 hours to 12 months, or a custom range) — the same range picker as the sibling Insights dashboards, so switching between them keeps your window.

Sort the breakdown table by volume. Your operating question for each language row is *"is this rate materially worse than the totals card?"* — judged against the org-wide rollup, not against English specifically. Rows with thin volume swing hard; look at the conversation count before trusting a single-digit escalation rate.

Three patterns and what they mean:

* **Low containment, normal resolution in one language** — the agent escalates, and humans resolve. The agent is giving up early in that locale; the knowledge and flows exist, the agent just can't reach them in that language.
* **High containment, low resolution** — the agent contains the customer but the conversation never resolves. The bot is looping in that language; containment without resolution is worse than an honest escalation.
* **Low CSAT or negative average sentiment where containment looks fine** — the language-specific prompts, templates, or handoff copy are register-wrong or literal-translated. Customers are being contained but leaving unhappy.

## Take action on a weak locale

Once you have named the pattern, the fixes are tenant-owned knobs — the same ones you already use for agent tuning, applied per language:

* **Localized prompt templates.** Add or refine prompt templates in the weak language so the agent reasons and answers natively instead of translating an English prompt. See [Prompt templates](/agents/prompt-templates).
* **Model routing.** Where a weaker-performing language tracks back to a model that handles it poorly, route that language's traffic to a model with stronger multilingual coverage. Your [agent model selection](/agents/model-selection) and [cost controls](/agents/cost-controls) settings decide which model a squad or agent serves.
* **Human queues for thin languages.** For a language with genuinely low volume, an intentional route-to-human can beat a poorly-contained bot: point the handoff at a queue or squad staffed for that locale ([Handoff targets](/agents/handoff-targets)).
* **Offline evals per language.** After a prompt- or model-level change, re-check the language row against the trailing window before the change. The [continuous production evals](/agents/continuous-production-evals) setup gives you the offline counterpart. Red team a localized variant the way you would a new prompt.

Verify the fix the same way you diagnosed it: watch the affected language's containment and CSAT rows converge toward the org totals over the next window, rather than re-reading the blended average.

## Combine with channel-language guidance

Per-language quality is the measurement layer for multilingual AI; the per-channel pages carry the language and locale handling rules for each channel — character sets, template languages, delivery quirks. When you roll a localized fix out, run through the channel pages your customers actually use — [WhatsApp](/channels/whatsapp), [SMS](/channels/sms), [LINE](/channels/line), [KakaoTalk](/channels/kakao), and the rest under **Channels** — so the localized behaviour is right end-to-end, not just inside the agent.

## API access

The endpoint is read-only and tenant-scoped:

```bash theme={null}
curl -s "https://api.orbit.devotel.io/api/v1/insights/language-quality?from=2026-08-01T00:00:00Z&to=2026-08-28T00:00:00Z" \
  -H "Authorization: Bearer $API_KEY"
```

Response shape: `from`/`to`/`timezone`, a `totals` rollup, a `languages` array (one row per detected language with the rates and averages above), plus `scanned_conversations` and `scan_truncated` so you can tell a full-window read from a most-recent sample. Pass `from`/`to` as ISO-8601 to scope the window (default: trailing 30 days); omitting both sides yields the full trailing window. For the full field listing see the [Insights API reference](/api-reference/insights).

## See also

* [Insights dashboards walkthrough](/guides/insights-dashboards) — the containment, sentiment, and agent-comparison surfaces this dashboard complements
* [Prompt templates](/agents/prompt-templates) — localized templates per language
* [Model selection](/agents/model-selection) — route weak locales to a stronger multilingual model
* [AI agent cost controls](/agents/cost-controls) — bound the cost of a per-language model split