Stream an Agent Response Token-by-Token over SSE

A blocking POST /agents/{id}/chat call makes your user stare at a spinner until the whole answer exists. The streaming endpoints put the first words on screen in under a second and keep them coming, which is what a chat-style UI needs to feel alive. This guide walks the consumer side end to end: picking a surface, parsing the wire format, cancelling mid-stream, recovering from a dropped connection, and folding token and cost totals into your ROI dashboards. Endpoint paths below are relative; send them against https://api.orbit.devotel.io/api/v1.

1. Why stream, and which surface to pick

Orbit ships two streaming surfaces for agent chat:

POST /agents/{id}/chat/stream — native token streaming. The API proxies the agent runtime’s SSE connection directly to you, so tokens arrive as the model produces them. Frames are bare data: lines: partial tokens as data: {"token":"…"}, and a terminal frame data: {"final":true,"conversation_id":"conv_…"}. Pick this whenever your runtime can hold an open HTTP response — it has the lowest time-to-first-token.
POST /agents/{id}/stream — word-chunked streaming. The API runs the blocking chat path, then re-emits the finished response word-by-word at roughly 30 ms gaps as named SSE events. Use it when an SDK or framework expects a text/event-stream response but cannot drive the native stream, or when you want the terminal done event to carry token and cost totals.

Both accept the same JSON body as POST /agents/{id}/chat: a required message, optional history (an array of {role, content} turns seeded additively on top of persisted state), and optional conversationId. If you only need the answer and not the UX, plain POST /agents/{id}/chat remains the simplest call.

2. Auth, quota, and conversation carry-forward

Auth. Send your secret API key in the X-API-Key header (dv_live_sk_…). The key needs the agents:write scope; both streaming surfaces are write actions, same as POST /chat.
Quota. All three chat surfaces — /chat, /chat/stream, and /stream — share one tenant budget of 20 requests per minute. Twenty concurrent chat sessions on one tenant queue against the same bucket, so a bursty UI can hit 429 even when each individual user is idle. Handle 429 explicitly (see §6).
Conversation carry-forward. Pass conversationId from the previous turn’s terminal frame to continue a conversation; omit it to start a fresh one. The id is the thread your whole UI hangs on — persist it the moment you receive it, not at the end of your handler.

3. Wire format

Every frame is standard SSE: optional event: line, a data: line, blank line. Keepalive comment frames (lines starting with :) may appear on quiet stretches; a correct reader ignores any line beginning with :. Native stream — POST /agents/{id}/chat/stream:

data: {"token":"Your"}

data: {"token":" refund"}

data: {"final":true,"conversation_id":"conv_9d4e2f1a"}

Tokens are partial chunks, not words — append them verbatim; do not insert spacing yourself. Word-chunked stream — POST /agents/{id}/stream:

event: token
data: {"text":"Your"}

event: token
data: {"text":" refund"}

event: done
data: {"totalTokens":412,"costCents":3,"conversationId":"conv_9d4e2f1a"}

Named events on this surface:

Event	Payload	Meaning
`token`	`{ text }`	One word; surrounding whitespace is preserved so concatenating every `text` reconstructs the answer exactly.
`done`	`{ totalTokens, costCents, conversationId }`	Terminal frame. Carries the usage totals and the conversation id to pass forward.
`error`	`{ code, message }`	Terminal frame for a failure after the stream opened (agent missing, model error).

An empty agent answer produces done with zero tokens and no token events — treat that as a valid, empty reply.

4. Consumers

curl

-N disables buffering so frames print as they arrive:

curl -N -X POST https://api.orbit.devotel.io/api/v1/agents/ag_123/stream \
  -H "X-API-Key: dv_live_sk_..." \
  -H "Content-Type: application/json" \
  -d '{"message":"Summarize today'\''s open tickets."}'

Node (native stream)

const res = await fetch("https://api.orbit.devotel.io/api/v1/agents/ag_123/chat/stream", {
  method: "POST",
  headers: { "X-API-Key": process.env.ORBIT_API_KEY, "Content-Type": "application/json" },
  body: JSON.stringify({ message: "Summarize today's open tickets." }),
});
if (!res.ok) throw new Error(`HTTP ${res.status}`); // 4xx/5xx arrive as JSON, not SSE

const reader = res.body.getReader();
const decoder = new TextDecoder();
let buf = "";
for (;;) {
  const { done, value } = await reader.read();
  if (done) break;
  buf += decoder.decode(value, { stream: true });
  let i;
  while ((i = buf.indexOf("\n\n")) >= 0) {
    const frame = buf.slice(0, i);
    buf = buf.slice(i + 2);
    if (frame.startsWith(":")) continue; // keepalive comment
    const data = frame.split("\n").find((l) => l.startsWith("data: "))?.slice(6);
    if (!data) continue;
    const payload = JSON.parse(data);
    if (payload.token) process.stdout.write(payload.token);
    if (payload.final) console.log("\nconversation:", payload.conversation_id);
  }
}

Browser — React `useEffect` (word-chunked stream)

Browsers cannot send a body through EventSource, so use a fetch reader. About thirty lines renders tokens as they land:

useEffect(() => {
  const controller = new AbortController();
  (async () => {
    const res = await fetch("/api/relay/agents/ag_123/stream", {
      method: "POST",
      headers: { "Content-Type": "application/json" },
      body: JSON.stringify({ message, conversationId }),
      signal: controller.signal,
    });
    const reader = res.body.getReader();
    const decode = new TextDecoder();
    let buf = "";
    for (;;) {
      const { done, value } = await reader.read();
      if (done) break;
      buf += decode.decode(value, { stream: true });
      let i;
      while ((i = buf.indexOf("\n\n")) >= 0) {
        const frame = buf.slice(0, i);
        buf = buf.slice(i + 2);
        if (frame.startsWith(":")) continue;
        const event = /^event: (.+)$/m.exec(frame)?.[1];
        const data = /^data: (.+)$/m.exec(frame)?.[1];
        if (!event || !data) continue;
        const payload = JSON.parse(data);
        if (event === "token") setText((t) => t + payload.text);
        if (event === "done") setConversationId(payload.conversationId);
        if (event === "error") setError(payload.message);
      }
    }
  })().catch(() => setError("Stream interrupted"));
  return () => controller.abort();
}, [message]);

Keep the API key on your server: in the browser, relay the stream through a thin backend route that adds X-API-Key, exactly as sketched above.

Python (httpx)

import json, sys
import httpx

with httpx.stream(
    "POST",
    "https://api.orbit.devotel.io/api/v1/agents/ag_123/stream",
    headers={"X-API-Key": "dv_live_sk_...", "Content-Type": "application/json"},
    json={"message": "Summarize today's open tickets."},
    timeout=None,
) as res:
    res.raise_for_status()
    event = None
    for line in res.iter_lines():
        if line.startswith(":"):
            continue  # keepalive comment
        if line.startswith("event: "):
            event = line[len("event: "):]
        elif line.startswith("data: "):
            payload = json.loads(line[len("data: "):])
            if event == "token":
                sys.stdout.write(payload["text"]); sys.stdout.flush()
            elif event == "done":
                print(f"\n[tokens={payload['totalTokens']} cost_cents={payload['costCents']} conversation={payload['conversationId']}]")
            elif event == "error":
                raise SystemExit(f"{payload['code']}: {payload['message']}")

Reconnecting after a dropped connection

The agent chat streams are request-scoped: one turn is one HTTP request, and tokens already emitted are not replayed. Orbit’s platform-event stream (GET /events/stream) is the surface with Last-Event-ID replay semantics — reconnect with the last seen id and buffered events newer than it are re-sent. The chat streams resume at the conversation level instead: start a new request carrying the same conversationId and the agent continues with its persisted history intact. Persist the id from every terminal frame (§2) so a dropped connection never loses the thread. For live dashboard data around the agent (message DLRs, call lifecycle), subscribe to /events/stream with Last-Event-ID and keep the chat stream dedicated to the reply itself.

5. Cancelling mid-stream

Call controller.abort() (browser), destroy the reader (Node), or close the response (Python). The server watches the socket and stops emitting the moment your connection closes — there is no separate cancel call. You are billed for tokens the model produced up to that point; cancellation stops further generation, it does not refund work already done.

6. Errors: `error` events versus HTTP status

Two failure classes, two handlers:

HTTP 4xx/5xx before the stream opens. The response is the standard JSON error envelope, not text/event-stream. Check res.ok first and branch on status: 401 bad or missing key, 403 missing agents:write scope, 404 unknown agent id, 422 body failed validation, 500 retry with backoff.
error event after the stream opened. A named event with { code, message }; treat it as terminal for that request.
429 quota exhausted. The shared 20/min budget returns 429 with a Retry-After header. Back off for that duration; do not retry immediately in a loop, and surface “the agent is busy” in the UI rather than failing the message silently. Spreading bursty traffic across the minute — or queueing turns client-side — beats parallel fan-out.

7. Cost telemetry from the `done` event

The word-chunked stream’s done event carries totalTokens and costCents for the turn. Accumulate them per agent id into the same ledger you feed your ROI dashboards — the agent ROI attribution guide shows the dashboard side of that picture. Two habits keep the numbers honest:

Record the totals from done, never by counting token events yourself: word chunks are display units, not billing units.
Store conversationId with each sample so cost rows join back to the conversation for debugging.

Quick reference

	`POST /agents/{id}/chat/stream`	`POST /agents/{id}/stream`
Framing	bare `data:` frames	named `event:` frames
Chunk unit	model tokens	words (~30 ms apart)
Terminal frame	`{final: true, conversation_id}`	`done` with `{totalTokens, costCents, conversationId}`
Mid-stream failure	stream closes	`error` event `{code, message}`
Auth	`X-API-Key` + `agents:write`	`X-API-Key` + `agents:write`
Rate limit	20 req/min per tenant, shared across the chat trio	same

WhatsApp

Developer console

Contact center

Guides

Messaging Controls

Audience & Support Tools

Stream an Agent Response Token-by-Token over SSE

Stream an Agent Response Token-by-Token over SSE

1. Why stream, and which surface to pick

2. Auth, quota, and conversation carry-forward

3. Wire format

4. Consumers

curl

Node (native stream)

Browser — React `useEffect` (word-chunked stream)

Python (httpx)

Reconnecting after a dropped connection

5. Cancelling mid-stream

6. Errors: `error` events versus HTTP status

7. Cost telemetry from the `done` event

Quick reference

​Stream an Agent Response Token-by-Token over SSE

​1. Why stream, and which surface to pick

​2. Auth, quota, and conversation carry-forward

​3. Wire format

​4. Consumers

​curl

​Node (native stream)

​Browser — React useEffect (word-chunked stream)

​Python (httpx)

​Reconnecting after a dropped connection

​5. Cancelling mid-stream

​6. Errors: error events versus HTTP status

​7. Cost telemetry from the done event

​Quick reference

Stream an Agent Response Token-by-Token over SSE

1. Why stream, and which surface to pick

2. Auth, quota, and conversation carry-forward

3. Wire format

4. Consumers

curl

Node (native stream)

Browser — React `useEffect` (word-chunked stream)

Python (httpx)

Reconnecting after a dropped connection

5. Cancelling mid-stream

6. Errors: `error` events versus HTTP status

7. Cost telemetry from the `done` event

Quick reference