​

Real-time agent-assist whisper coaching in the softphone

​

Where the panel lives

• suggested_response — one short paragraph the agent could say next, written to be spoken aloud (no greetings or sign-offs; the call is in progress).
• kb_articles — up to three articles retrieved from your tenant’s own knowledge base. The model only distils the caller’s current informational need into a search query; the surfaced titles and snippets always come from real documents, each with document_id (and source_url where the source carries one) so the agent can open the article, never a hallucinated citation.
• next_best_action — a single imperative next step (“offer a $20 credit”, “escalate to billing tier 2”).
• suggested_action — where the next best step maps to a whitelisted one-click action (today: escalating to a ticket), a structured action object so the UI renders an execute button instead of plain text.
• sentiment_score — a rolling [-1, +1] estimate of the caller’s current tone in the most recent turns, never an average over the whole call. Most ordinary conversations sit between −0.3 and +0.3, so treat −0.5 as genuinely unhappy, not merely quiet.

​

Consuming the stream end-to-end over SSE

curl -N "https://orbit.devotel.io/api/v1/voice/calls/call_abc123/agent-assist/stream" \
  -H "X-API-Key: dv_live_sk_your_key_here" \
  -H "Accept: text/event-stream"

1. event: connected — handshake with { "callId", "status" }.
2. event: status — { "status", "terminal": true|false }. If the call is already in a terminal state (completed, failed, busy, no-answer, canceled), the stream sends event: end with { "reason": "terminal" } and closes cleanly — connect and subscribe, don’t poll for cards that will never come.
3. event: assist — one frame per card. The data is the card JSON plus a server-issued envelope: monotonically increasing id, generated_at timestamp, and the model name.
4. : ping — a comment-only keepalive line every 15 seconds so proxies and browsers see traffic even when no card is due.
5. event: end — { "reason": "terminal" } when the call ends, or { "reason": "max-duration" } at the 30-minute hard cap; the client is expected to reconnect and continue from its last received id.

import EventSource from "eventsource"; // or the browser-native EventSource

interface AssistCard {
  id: number;
  generated_at: string;
  model: string;
  suggested_response: string;
  kb_articles: Array<{
    title: string;
    snippet?: string;
    document_id?: string;
    source_url?: string;
  }>;
  next_best_action: string;
  suggested_action?: {
    tool: "escalate_to_ticket";
    args: { reason: string };
  };
  sentiment_score?: number;
  confidence?: number;
}

function readStream(callId: string, apiKey: string): void {
  const es = new EventSource(
    `https://orbit.devotel.io/api/v1/voice/calls/${callId}/agent-assist/stream`,
    { headers: { "X-API-Key": apiKey } },
  );

  // The server emits `: ping` comment keepalives every 15 s, but the
  // standard EventSource API never surfaces comment lines to
  // listeners — absence of `assist` frames alone therefore means
  // "no new card", not "dead stream". Detect a stalled socket at the
  // transport layer (fetch + manual parse) if you need a heartbeat;
  // otherwise let `onerror`/`onclose` drive reconnection.
  es.onerror = () => {
    es.close();
    reconnect(callId);
  };

  es.addEventListener("connected", (ev) => {
    console.log("open:", JSON.parse(ev.data));
  });

  es.addEventListener("status", (ev) => {
    const { status, terminal } = JSON.parse(ev.data);
    if (terminal) es.close(); // ended call — nothing will arrive
    console.log("status:", status, terminal);
  });

  es.addEventListener("assist", (ev) => {
    const card: AssistCard = JSON.parse(ev.data);
    renderCard(card); // your UI hook — see "Building a custom coach widget"
  });

  es.addEventListener("end", (ev) => {
    const { reason } = JSON.parse(ev.data);
    es.close();
    if (reason === "max-duration") reconnect(callId); // keep watching
    else console.log("done");
  });
}

​

Tuning thresholds, debounce, and the transcript window

• Cadence gate — one card at most every 8 seconds and only after at least 3 new final transcript turns; a fast speaker produces a card roughly every three turns, a slow one never quicker than eight seconds. You tune cadence on the client side instead — render or discard, don’t re-fetch — so dashboards can show every fifth card or pin a favourite card without changing the stream.
• Transcript window — the model sees the trailing ~6,000 characters (about a page and a half of speech) per card. The window never grows unbounded, so a 60-minute call costs the same per card as a 6-minute one. If your workflow needs long-range context (cross-call history, CRM records), fetch it in your own client and merge it with the cards — the endpoint intentionally carries only the rolling window.
• Sentiment alarm threshold — the alert fires the moment the rolling score drops to −0.5 or below, the same boundary the post-call sentiment alerting uses. Keep per-agent per-call sensitivity on the client: e.g. suppress the supervisor nudge for an agent until the score stays below the threshold for two consecutive cards, or mark −0.8 as your own barge redline.

​

Building a custom coach widget

function renderCard(card: AssistCard): void {
  list.appendChild(
    el("article", [
      el("p", card.suggested_response),
      card.kb_articles.map((a) =>
        el("a", { href: a.source_url }, a.title),
      ),
      el("strong", card.next_best_action),
    ]),
  );
}

• When suggested_action is present, render a button. Today’s only whitelisted tool is escalate_to_ticket; press the button and the UI calls the existing ticket-creation endpoint with the bounded args.reason (max 300 characters), never a raw client-side tool invocation. When suggested_action is absent the next-best-action line stays plain text — never invent an execute button for unlisted actions.
• Colour the sentiment strip by threshold: neutral above −0.5, warning at or below −0.5, your own team’s redline (often −0.8) below. The score is a rolling estimate of the most recent turns, so it can swing; smooth on the client with a short EMA if the strip flickers.

​

Supervisor lane: whisper vs barge

• The supervisor wallboard gets a voice.sentiment.negative event with a recommended action: whisper (coach the agent silently) for a score of −0.7 or higher, barge (join audibly) below −0.7. Cadence is deliberately throttled: at most one alert per call per minute, so a persistent angry stretch lights the wallboard once, not once per card.
• The agent’s own overlay gets an automated calming nudge instead of relying on the manual distress button.

​

Failure modes and unhealthy streams

• Card generation blows up — the call, transcript, and stream are untouched; the panel simply skips a tick. Diagnose by watching for a stream that opens connected and keeps : pinging but produces no assist frames for longer than a minute.
• The call is already over when you connect — you get event: status with terminal: true, then event: end { "reason": "terminal" }. Reconnecting is pointless; use the post-call QA scorecard endpoints instead.
• The stream goes silent — : ping comment keepalives run every 15 seconds, but the standard EventSource API suppresses comment lines, so a stalled socket shows up only as an onerror/close (or a transport-level parser) signal, not “no frames”. Treat that as a dead proxy or a pod restart — and rebuild — rather than as “no cards due”.
• The 30-minute cap elapses — you get event: end { "reason": "max-duration" }; reconnect with the same URL, no resume token needed, and continue from the last received id.
• Access wrong — cross-tenant or missing call ids return 404 before the stream opens; missing or insufficient credentials return 401/403 in the same envelope — test for the HTTP error before wiring the render loop.

​

Related

• Supervisor live monitoring for voice calls — listen, whisper, barge, conference-join
• Call QA scorecards and auto-scoring — grade the same calls after the live coaching lane has done its work
• Live transcript SSE — agent-assist and supervisor-assist wire formats