Skip to main content

​
Live interpretation in video rooms

Live interpretation is the spoken counterpart of the translated-captions line. The captions surface renders a translated caption as on-screen text; the interpretation surface returns a line your client renders as speech on a dedicated per-language audio track — the interpreter channel a participant listens to instead of the floor audio. A room can carry several interpreter channels at once — one per language — alongside the translated-captions surface, which is screen text rather than speech. Each channel is its own language direction: French speakers interpreted into English, English speakers interpreted into French. The Authoring-side piece is one endpoint, POST /video/interpretation, that turns one finalized utterance into the interpreted line for one channel. This guide covers how the channel works, how to drive the endpoint, what joining mid-meeting means, what is retained, and how usage is billed.

​
What a live-interpretation session is

An interpreter channel is not a separate room or a separate participant — it is an additional audio track inside the room, overlaid by the participant’s client:
  1. The room’s caption pipeline already produces finalized utterances per speaker (browser → speech-to-text → the room’s data channel).
  2. For each finalized utterance, a participant’s client posts the utterance text, the speaker’s language, and the channel’s target language to POST /video/interpretation.
  3. The response carries translated_text — natural spoken phrasing, not a literal gloss. The client hands that line to text-to-speech and renders it onto the per-language interpretation track.
  4. Listeners subscribed to that channel hear the interpreted line instead of the original audio.
The interpretation is machine interpretation under your organization’s scope — every call is tenant-scoped to your organization, billed to your wallet, and gated by your LLM spend caps. There is no external human interpreter pool behind this endpoint, so availability does not depend on booking interpreters ahead of time; it depends only on your wallet having spend headroom. (Routing a human interpreter into a room is something you do at the room layer — invite them as a participant and let them publish their own audio — not through this endpoint.)
Live interpretation is speech. If you only need on-screen translated text, use the translated-captions surface instead — same utterance pipeline, no text-to-speech step.

​
Language pairs, session participants, and late join

Language pairs. Both source_language and target_language accept the same closed set of language codes: en, es, fr, de, hi, it, ja, ko, nl, pt, ru, sv, tr, zh Declare the speaker’s language in source_language and the channel’s language in target_language. When the two are equal — a French-speaking participant on the French channel — the endpoint short-circuits: it returns interpreted: false with the original text echoed back, makes no billable model call, and your client passes the original audio through. Only send an utterance to the channels whose language differs from the speaker’s. Who can drive the channel. The endpoint requires an authenticated caller with the owner, admin, or developer role and the video:write scope. In practice that is the participant client (or room orchestrator) running under credentials the same role-holding user already uses to join and publish in the room — see Video room access tokens for the join credentials themselves. A video:read-only key can probe the route but cannot run billable interpretation; it receives a 403 on the write. Late join. Interpretation is stateless per utterance — there is no session to open or close, and a participant who joins the room mid-meeting starts hearing their channel from the next finalized utterance onward. Nothing is buffered server-side to replay; if a late joiner needs what was said before they arrived, the post-meeting transcript and summary pipeline is the surface that carries history.

​
Driving the interpreter channel

Post one finalized utterance per call. The body is strict — unknown fields are rejected — and text is capped at 2,000 characters (a finalized utterance is far shorter; the cap exists so a malformed client cannot push a large blob into a billable call):
On the response:
  • interpreted: true — the utterance was interpreted; render translated_text with text-to-speech onto the channel track.
  • interpreted: false — same-language short-circuit; translated_text equals source_text, no model was called, and no usage accrued. Pass the original audio through.
  • speaker_name — echoed verbatim so you can attribute the line on the channel without a second lookup.
  • model — the current fast-model id that produced the line (null on the short-circuit). Treat it as informational; do not pin prompts or costs to a specific model id.
Failure modes to handle in the client:
  • 400 INVALID_LANGUAGE — a language outside the supported set. The error message lists the accepted codes; fix the channel configuration, do not retry.
  • 400 INVALID_REQUEST — the body failed validation (missing text, over-cap characters, unknown fields). The response carries the per-field issues; correct and retry.
  • Sanitized 5xx envelope — the model call failed. The endpoint is off the room’s hot path: a failed interpretation call must never block audio. Skip that utterance on the channel and continue with the next one.

​
Transcripts and retention — what you own

The interpretation hop persists nothing. The source utterance and the interpreted line stay on the ephemeral path — browser → speech-to-text → the room’s data channel → your client → text-to-speech — and the audit entry the endpoint writes records only the language pair, the model, and the utterance’s character count, never the spoken content. There is no per-line interpretation store to set a window on. What is retained lives in the existing video and retention surfaces, and the controls are yours:
  • Caption/transcript buffers — the host client buffers live captions during the meeting and flushes them at room-end into the session’s post- meeting pipeline (transcript, summary, QA scoring). That buffer is the only record of what was said, and it is the customer’s to supply or skip.
  • Room recordings — if the room is recorded, the recording is governed by your recording retention window; the interpretation track’s speech is part of the mix your client renders into the room.
  • Video room sessions — the session-history row itself follows the video-room window.
Windows, redaction-versus-purge semantics, and the legal-hold and export overrides are covered in Retention windows and deletion and configured on the data retention policy page. If you need a spoken interpretation record at all, decide it at the recording layer — the endpoint itself gives you no transcript to retain.

​
Billing and usage

Each interpreted utterance is one billable model call against your organization’s wallet, subject to the same daily spend caps and rate gates as every other AI feature in the workspace. Two consequences to build against:
  • When spend headroom runs out, interpretation calls start failing — the room itself is unaffected. Monitor your LLM spend the same way you do for chat-message translation and caption translation; they draw from the same ledger.
  • The same-language short-circuit (interpreted: false) is free — it never reaches the model. Sending utterances only to channels whose language differs from the speaker’s is both the correct behavior and the cheapest.
Interpreted-line volume also shows up in your per-record usage feed. See Usage records: the per-record billing feed for how to reconcile feature usage back to the wallet ledger, and AI spend attribution for the dashboard view.

​
Where to go next