Skip to main content

​
Broadcast captions and the translation model

Orbit renders live captions in two places: in-room, over the LiveKit data channel, and on the external broadcast egress (HLS/RTMP) as a selectable WebVTT subtitle track. Translation sits on top of the same pipeline — each viewer’s client asks the platform to render one finalized caption line in their own language. This page covers how those two surfaces are built, why the translation hop is deliberately stateless, and what happens when each stage degrades. The room model and recording pipeline are covered in The video room model.

​
Captions versus live interpretation

Two surfaces borrow from the same speech-to-text pipeline; they are distinct features. Captions are the textual surface this page covers. Live interpretation is the spoken-interpreter channel — the audio analog of a translated caption line — and lives alongside it in the room.

​
Ephemeral-line design: browser → Deepgram → LiveKit

Transcription never touches your Orbit transcript storage in real time. The flow is browser-side end to end:
  1. The room host enables live captions (a deploy-level capability probe, GET /video/captions/capabilities, hides the toggle when it is not configured).
  2. The enabled participant’s browser mints a 5-minute ephemeral Deepgram key (POST /video/captions/ephemeral-key) and opens a WebSocket straight to Deepgram. Your long-lived Deepgram credential never reaches the browser; the worst-case leak is a 5-minute STT budget. The client refreshes the key ~150 s before expiry.
  3. Deepgram finals are published by the speaker into the LiveKit data channel. Only finals cross the wire — partial (interim) lines churn at 5–10 Hz per speaker and would contend with chat and reaction traffic, so remote viewers get stable, settled lines only.
  4. Redaction (pci, pii, numbers) is applied as query parameters on the Deepgram WebSocket URL using the server-emitted policy, and the mint is audit-logged with the policy and language. This is defense-in-depth: the durable enforcement is project-level redaction in your Deepgram dashboard (operator action) plus the audit record.
Nothing written this way is stored server-side; the transcript is the room’s product and stays in the room. Durable transcripts are built post-meeting from the recording egress.

​
The translation request and response

A viewer translates one finalized line at a time by posting it with the language they want to read:
  • 2000-character cap. A single caption line is capped at 2000 characters. Deepgram final segments are far shorter; the ceiling exists so a malformed or hostile client cannot push a multi-kilobyte blob into a billable LLM call. The same cap bounds the per-segment webhook (POST /video/captions/:name/segment) so the whole pipeline is uniformly bounded.
  • No-op short-circuit. When the client supplies the speaker’s transcription language and it equals the viewer’s target, the response echoes the line back (model: null) with no LLM call and no spend.
Requests pass an owner/admin/developer role gate and the video:write scope; successful calls are audit-logged with lengths only — caption text itself (which may carry meeting PII) is never written to the audit chain.

​
The closed target-language set

The target_language enum is closed, not free-form. It deliberately excludes Deepgram’s multi sentinel: multi is a single-stream code-switching transcription mode — a valid input language for transcription on the ephemeral-key endpoint — not a language you can translate text into. A free-form field would let a browser request arbitrary targets and spend against them; the whitelist keeps the LLM-spend surface auditable and makes a future language addition a one-line edit paired with a regression test. Supported targets: en, es, fr, de, hi, it, ja, ko, nl, pt, ru, sv, tr, zh.

​
LLM spend routing

One line translation is a deliberately small LLM call on the tenant’s fast model (a Claude fast-tier model resolved at request time, never hardcoded). It reuses the exact plumbing of chat-message translation, so the spend lands on the tenant wallet and the cost ledger with the same shape, and it obeys the same gates:
  • Tenant wallet debit per call.
  • Daily cap on LLM spend.
  • Tokens-per-minute (TPM) throttle.
  • Circuit breaker on an over-budget or unhealthy model.
Because every hop is metered, a runaway viewer or a compromised client cannot burn unbounded translation spend through this endpoint.

​
Why the pipeline is deliberately stateless

Chat-message translation caches results under the message row, because a chat message is a durable record likely to be read again. A live caption line is ephemeral and effectively unique per utterance; caching would add a DB write path and storage for a string that is never requested twice. The endpoint therefore returns cached: false and performs no DB read or write — statelessness is the design, not a gap to backfill. For broadcast consumers the rolling Redis window is the counterpart: it is a playback-text surface with a 6-hour TTL, not a transcript store.

​
Failure modes and the viewer fallback

Every stage of the pipeline degrades to a clear failure, never a 500 that stalls the meeting:

​
Diagram

The video room model

Room creation, lifecycle, participation tiers, and where this feature attaches.

Recording lifecycle

How post-meeting transcripts are built from egress — the durable counterpart of the ephemeral caption line.

Avatar bot dispatch queue

How avatar joins run through the same LiveKit room pipeline.

LLM spend cost attribution

Wallet routing, daily caps, TPM throttles, and circuit breakers.

Synthetic-media provenance

Provenance signing for rooms using avatars and rendered content.

Video API reference

The endpoint-by-endpoint contract this model implements.