> ## Documentation Index
> Fetch the complete documentation index at: https://docs.orbit.devotel.io/llms.txt
> Use this file to discover all available pages before exploring further.

# Stream an Agent Response Token-by-Token over SSE

> Consume the agent streaming endpoints from Node, the browser, and Python: wire format, cancellation, reconnect strategy, quota behaviour, and reading token and cost totals from the terminal event.

# Stream an Agent Response Token-by-Token over SSE

A blocking `POST /agents/{id}/chat` call makes your user stare at a spinner
until the whole answer exists. The streaming endpoints put the first words
on screen in under a second and keep them coming, which is what a
chat-style UI needs to feel alive. This guide walks the consumer side end
to end: picking a surface, parsing the wire format, cancelling mid-stream,
recovering from a dropped connection, and folding token and cost totals
into your ROI dashboards.

Endpoint paths below are relative; send them against
`https://api.orbit.devotel.io/api/v1`.

## 1. Why stream, and which surface to pick

Orbit ships two streaming surfaces for agent chat:

* `POST /agents/{id}/chat/stream` — **native token streaming.** The API
  proxies the agent runtime's SSE connection directly to you, so tokens
  arrive as the model produces them. Frames are bare `data:` lines:
  partial tokens as `data: {"token":"…"}`, and a terminal frame
  `data: {"final":true,"conversation_id":"conv_…"}`. Pick this whenever
  your runtime can hold an open HTTP response — it has the lowest
  time-to-first-token.
* `POST /agents/{id}/stream` — **word-chunked streaming.** The API runs the
  blocking chat path, then re-emits the finished response word-by-word at
  roughly 30 ms gaps as named SSE events. Use it when an SDK or framework
  expects a `text/event-stream` response but cannot drive the native
  stream, or when you want the terminal `done` event to carry token and
  cost totals.

Both accept the same JSON body as `POST /agents/{id}/chat`: a required
`message`, optional `history` (an array of `{role, content}` turns seeded
additively on top of persisted state), and optional `conversationId`.
If you only need the answer and not the UX, plain `POST /agents/{id}/chat`
remains the simplest call.

## 2. Auth, quota, and conversation carry-forward

* **Auth.** Send your secret API key in the `X-API-Key` header
  (`dv_live_sk_…`). The key needs the `agents:write` scope; both streaming
  surfaces are write actions, same as `POST /chat`.
* **Quota.** All three chat surfaces — `/chat`, `/chat/stream`, and
  `/stream` — share one tenant budget of **20 requests per minute**.
  Twenty concurrent chat sessions on one tenant queue against the same
  bucket, so a bursty UI can hit `429` even when each individual user is
  idle. Handle `429` explicitly (see §6).
* **Conversation carry-forward.** Pass `conversationId` from the previous
  turn's terminal frame to continue a conversation; omit it to start a
  fresh one. The id is the thread your whole UI hangs on — persist it the
  moment you receive it, not at the end of your handler.

## 3. Wire format

Every frame is standard SSE: optional `event:` line, a `data:` line, blank
line. Keepalive comment frames (lines starting with `:`) may appear on
quiet stretches; a correct reader ignores any line beginning with `:`.

**Native stream — `POST /agents/{id}/chat/stream`:**

```text theme={null}
data: {"token":"Your"}

data: {"token":" refund"}

data: {"final":true,"conversation_id":"conv_9d4e2f1a"}

```

Tokens are partial chunks, not words — append them verbatim; do not insert
spacing yourself.

**Word-chunked stream — `POST /agents/{id}/stream`:**

```text theme={null}
event: token
data: {"text":"Your"}

event: token
data: {"text":" refund"}

event: done
data: {"totalTokens":412,"costCents":3,"conversationId":"conv_9d4e2f1a"}

```

Named events on this surface:

| Event | Payload | Meaning |
| - | - | - |
| `token` | `{ text }` | One word; surrounding whitespace is preserved so concatenating every `text` reconstructs the answer exactly. |
| `done` | `{ totalTokens, costCents, conversationId }` | Terminal frame. Carries the usage totals and the conversation id to pass forward. |
| `error` | `{ code, message }` | Terminal frame for a failure after the stream opened (agent missing, model error). |

An empty agent answer produces `done` with zero tokens and no `token`
events — treat that as a valid, empty reply.

## 4. Consumers

### curl

`-N` disables buffering so frames print as they arrive:

```bash theme={null}
curl -N -X POST https://api.orbit.devotel.io/api/v1/agents/ag_123/stream \
  -H "X-API-Key: dv_live_sk_..." \
  -H "Content-Type: application/json" \
  -d '{"message":"Summarize today'\''s open tickets."}'
```

### Node (native stream)

```js theme={null}
const res = await fetch("https://api.orbit.devotel.io/api/v1/agents/ag_123/chat/stream", {
  method: "POST",
  headers: { "X-API-Key": process.env.ORBIT_API_KEY, "Content-Type": "application/json" },
  body: JSON.stringify({ message: "Summarize today's open tickets." }),
});
if (!res.ok) throw new Error(`HTTP ${res.status}`); // 4xx/5xx arrive as JSON, not SSE

const reader = res.body.getReader();
const decoder = new TextDecoder();
let buf = "";
for (;;) {
  const { done, value } = await reader.read();
  if (done) break;
  buf += decoder.decode(value, { stream: true });
  let i;
  while ((i = buf.indexOf("\n\n")) >= 0) {
    const frame = buf.slice(0, i);
    buf = buf.slice(i + 2);
    if (frame.startsWith(":")) continue; // keepalive comment
    const data = frame.split("\n").find((l) => l.startsWith("data: "))?.slice(6);
    if (!data) continue;
    const payload = JSON.parse(data);
    if (payload.token) process.stdout.write(payload.token);
    if (payload.final) console.log("\nconversation:", payload.conversation_id);
  }
}
```

### Browser — React `useEffect` (word-chunked stream)

Browsers cannot send a body through `EventSource`, so use a `fetch`
reader. About thirty lines renders tokens as they land:

```tsx theme={null}
useEffect(() => {
  const controller = new AbortController();
  (async () => {
    const res = await fetch("/api/relay/agents/ag_123/stream", {
      method: "POST",
      headers: { "Content-Type": "application/json" },
      body: JSON.stringify({ message, conversationId }),
      signal: controller.signal,
    });
    const reader = res.body.getReader();
    const decode = new TextDecoder();
    let buf = "";
    for (;;) {
      const { done, value } = await reader.read();
      if (done) break;
      buf += decode.decode(value, { stream: true });
      let i;
      while ((i = buf.indexOf("\n\n")) >= 0) {
        const frame = buf.slice(0, i);
        buf = buf.slice(i + 2);
        if (frame.startsWith(":")) continue;
        const event = /^event: (.+)$/m.exec(frame)?.[1];
        const data = /^data: (.+)$/m.exec(frame)?.[1];
        if (!event || !data) continue;
        const payload = JSON.parse(data);
        if (event === "token") setText((t) => t + payload.text);
        if (event === "done") setConversationId(payload.conversationId);
        if (event === "error") setError(payload.message);
      }
    }
  })().catch(() => setError("Stream interrupted"));
  return () => controller.abort();
}, [message]);
```

Keep the API key on your server: in the browser, relay the stream through
a thin backend route that adds `X-API-Key`, exactly as sketched above.

### Python (httpx)

```python theme={null}
import json, sys
import httpx

with httpx.stream(
    "POST",
    "https://api.orbit.devotel.io/api/v1/agents/ag_123/stream",
    headers={"X-API-Key": "dv_live_sk_...", "Content-Type": "application/json"},
    json={"message": "Summarize today's open tickets."},
    timeout=None,
) as res:
    res.raise_for_status()
    event = None
    for line in res.iter_lines():
        if line.startswith(":"):
            continue  # keepalive comment
        if line.startswith("event: "):
            event = line[len("event: "):]
        elif line.startswith("data: "):
            payload = json.loads(line[len("data: "):])
            if event == "token":
                sys.stdout.write(payload["text"]); sys.stdout.flush()
            elif event == "done":
                print(f"\n[tokens={payload['totalTokens']} cost_cents={payload['costCents']} conversation={payload['conversationId']}]")
            elif event == "error":
                raise SystemExit(f"{payload['code']}: {payload['message']}")
```

### Reconnecting after a dropped connection

The agent chat streams are **request-scoped**: one turn is one HTTP
request, and tokens already emitted are not replayed. Orbit's
platform-event stream (`GET /events/stream`) is the surface with
`Last-Event-ID` replay semantics — reconnect with the last seen `id` and
buffered events newer than it are re-sent. The chat streams resume at the
conversation level instead: start a new request carrying the same
`conversationId` and the agent continues with its persisted history
intact. Persist the id from every terminal frame (§2) so a dropped
connection never loses the thread. For live dashboard data around the
agent (message DLRs, call lifecycle), subscribe to `/events/stream` with
`Last-Event-ID` and keep the chat stream dedicated to the reply itself.

## 5. Cancelling mid-stream

Call `controller.abort()` (browser), destroy the reader (Node), or close
the response (Python). The server watches the socket and stops emitting
the moment your connection closes — there is no separate cancel call. You
are billed for tokens the model produced up to that point; cancellation
stops further generation, it does not refund work already done.

## 6. Errors: `error` events versus HTTP status

Two failure classes, two handlers:

* **HTTP 4xx/5xx before the stream opens.** The response is the standard
  JSON error envelope, not `text/event-stream`. Check `res.ok` first and
  branch on status: `401` bad or missing key, `403` missing `agents:write`
  scope, `404` unknown agent id, `422` body failed validation, `500`
  retry with backoff.
* **`error` event after the stream opened.** A named event with
  `{ code, message }`; treat it as terminal for that request.
* **`429` quota exhausted.** The shared 20/min budget returns `429` with a
  `Retry-After` header. Back off for that duration; do not retry
  immediately in a loop, and surface "the agent is busy" in the UI rather
  than failing the message silently. Spreading bursty traffic across the
  minute — or queueing turns client-side — beats parallel fan-out.

## 7. Cost telemetry from the `done` event

The word-chunked stream's `done` event carries `totalTokens` and
`costCents` for the turn. Accumulate them per agent id into the same
ledger you feed your ROI dashboards — the
[agent ROI attribution guide](/guides/agent-roi-attribution) shows the
dashboard side of that picture. Two habits keep the numbers honest:

* Record the totals from `done`, never by counting `token` events
  yourself: word chunks are display units, not billing units.
* Store `conversationId` with each sample so cost rows join back to the
  conversation for debugging.

## Quick reference

| | `POST /agents/{id}/chat/stream` | `POST /agents/{id}/stream` |
| - | - | - |
| Framing | bare `data:` frames | named `event:` frames |
| Chunk unit | model tokens | words (\~30 ms apart) |
| Terminal frame | `{final: true, conversation_id}` | `done` with `{totalTokens, costCents, conversationId}` |
| Mid-stream failure | stream closes | `error` event `{code, message}` |
| Auth | `X-API-Key` + `agents:write` | `X-API-Key` + `agents:write` |
| Rate limit | 20 req/min per tenant, shared across the chat trio | same |


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.