> ## Documentation Index
> Fetch the complete documentation index at: https://docs.orbit.devotel.io/llms.txt
> Use this file to discover all available pages before exploring further.

# Configure event sinks: Kafka or HTTP batch

> Choose between the Kafka and HTTP-batch event sinks, configure either destination with its credential, understand batching and retry semantics, and inspect delivery health — plus full Node.js and Python consumer examples.

# Configure event sinks: Kafka or HTTP batch

An event sink streams the entire platform event taxonomy — the same `message.*`, `call.*`, … events your webhooks receive — to one destination you run: a Kafka topic, or an HTTP collector that receives batched JSON arrays. One sink replaces a fleet of single-event webhook endpoints. This guide walks through choosing a transport, configuring it in the dashboard or through the API, and operating it in production.

## What event sinks are vs webhooks

Both surfaces are push-based: Orbit delivers events to you rather than you polling for them. The difference is the destination shape:

* **Webhooks** — per-endpoint HTTPS POSTs, one event per request, with an `X-Orbit-Signature` HMAC over each body. You register endpoints on an event list and each becomes a steeply-secured receiver.
* **Event sinks** — a single declared destination per transport (`kafka` or `http_batch`); a subscriber filter such as `["message.*"]` decides which of the platform's event taxonomy lands there. The Kafka transport is not an HTTP receiver at all, and the HTTP-batch transport gets a JSON **array** of events in one POST.

Choose sinks when you want one destination carrying many event types, when you already run Kafka, or when per-POST HMAC signing is wasteful against your consumer code (sink records are byte-identical to the webhook envelope — one parser serves both). Choose webhooks when a third-party expects a single-purpose URL or when you want per-endpoint signing without any consumer library. The full model is in [Webhook fan-out](/concepts/webhook-fan-out-and-event-sinks); the live transport contract is in [Webhooks overview](/webhooks/overview).

## Choose destination — HTTP batch or Kafka

* **HTTP batch** fits a receiver you can stand up behind HTTPS: a Lambda-flavored endpoint, a log-collecting pipeline, an internal orchestration layer that accepts arrays. Tuning knob is `max_batch_size` (1–1000 events per POST body, default 100).
* **Kafka** fits a data plane that already runs brokers: you give Orbit the bootstrap brokers, a topic, and an optional SASL credential; Orbit produces one record per event keyed by your partition field so per-contact ordering holds.

Both sinks are independent — you can run either, both, or neither. If neither is enabled, nothing is delivered to sinks and your webhooks still work.

## Configure HTTP batch

Configure it in the dashboard under **Developer → Event Sinks → HTTP batch**, or through `PATCH /api/v1/developer/event-sinks/http_batch`:

```bash theme={null}
curl -X PATCH "https://api.orbit.devotel.io/api/v1/developer/event-sinks/http_batch" \
  -H "X-API-Key: dv_live_sk_your_key_here" \
  -H "Content-Type: application/json" \
  -d '{
    "enabled": true,
    "events": ["message.*", "call.completed"],
    "destination": {
      "kind": "http_batch",
      "url": "https://collector.example.com/orbit/events",
      "max_batch_size": 200
    },
    "credentials": "a-bearer-token"
  }'
```

The collector `url` must be `https` and resolve to a public address — private and metadata hosts are rejected, which is also why a `PATCH` that fails your organization's ingress proxy can't be smuggled in. The `credentials` value becomes the `Authorization: Bearer` header on every batch POST; it is encrypted at rest and never returned (`credentials_configured: true` is your proof it stored). Retry behaviour is covered below.

### Node.js consumer for HTTP batch (payload-rich envelopes)

```javascript theme={null}
import express from "express";

const app = express();
app.use(express.json({ limit: "2mb" })); // batches arrive as JSON arrays

app.post("/orbit/events", (req, res) => {
  // Optional: verify the Bearer token you configured as the sink credential.
  if (req.headers.authorization !== `Bearer ${process.env.ORBIT_SINK_TOKEN}`) {
    return res.status(401).json({ error: "Bad token" });
  }
  const events = req.body; // always an array, up to max_batch_size entries
  for (const event of events) {
    switch (event.type) {
      case "message.delivered":
        markDelivered(event.data.message_id);
        break;
      case "message.failed":
        flagFailure(event.data.message_id, event.data.error_code);
        break;
      case "call.completed":
        recordCallOutcome(event.data.call_id);
        break;
    }
    // Dedupe on event.id — at-least-once means replays are re-deliveries.
  }
  return res.status(200).json({ received: events.length });
});
```

The body is a plain JSON array; every entry is the same `id` / `type` / `created_at` / `data` envelope the webhook page documents. Ack the whole batch with `2xx` — a non-2xx makes the sink retry the batch with exponential backoff, which is why you dedupe on `event.id` at the receiver.

## Configure Kafka

In the dashboard under **Developer → Event Sinks → Kafka**, or through `PATCH /api/v1/developer/event-sinks/kafka`:

```bash theme={null}
curl -X PATCH "https://api.orbit.devotel.io/api/v1/developer/event-sinks/kafka" \
  -H "X-API-Key: dv_live_sk_your_key_here" \
  -H "Content-Type: application/json" \
  -d '{
    "enabled": true,
    "events": ["*"],
    "destination": {
      "kind": "kafka",
      "topic": "orbit.events",
      "partition_key_path": "data.contact_id"
    },
    "brokers": ["broker-1.example.com:9092", "broker-2.example.com:9092"],
    "sasl_mechanism": "scram-sha-512",
    "sasl_username": "orbit-producer",
    "credentials": "the-sasl-password"
  }'
```

The four required connection inputs:

* `brokers` — 1–16 bootstrap brokers as `host:port`, no scheme, no path.
* `destination.topic` — the Kafka topic to produce to; 1–249 chars, `[A-Za-z0-9._-]`, and never `.` or `..`.
* `destination.partition_key_path` — optional. A dot-path into the event envelope used to derive the record's partition key. Default chain: `data.contact_id` → `data.conversation_id` → `data.message_id` → `data.call_id` → `data.to` → event `id`. Co-locating one contact's events on one partition is the ordering guarantee downstream consumers can rely on.
* `sasl_mechanism` + `sasl_username` + `credentials` — the SASL handshake. Mechanisms: `plain`, `scram-sha-256`, `scram-sha-512`. The password is the encrypted credential; the username is stored plaintext alongside the brokers. If your brokers are not SASL-protected, omit all three and the sink produces without authentication.

`destination.kind` must equal the path segment — `PATCH /…​/kafka` with `destination.kind: "http_batch"` is a `400`.

### Python consumer (confluent-kafka)

```python theme={null}
import json
from confluent_kafka import Consumer

c = Consumer({
    "bootstrap.servers": "broker-1.example.com:9092",
    "group.id": "orbit-sink-consumer",
    "auto.offset.reset": "earliest",
    "security.protocol": "SASL_SSL",
    "sasl.mechanism": "SCRAM-SHA-512",
    "sasl.username": "orbit-producer",
    "sasl.password": "the-sasl-password",
})
c.subscribe(["orbit.events"])

while True:
    msg = c.poll(1.0)
    if msg is None:
        continue
    if msg.error():
        continue
    event = json.loads(msg.value().decode("utf-8"))
    # event["id"] — dedupe on this; at-least-once redelivers under retry.
    # msg.key() — the partition key (e.g. the contact_id), utf-8 bytes.
    # event["type"], event["created_at"], event["data"] — same envelope as webhooks.
    headers = dict(msg.headers() or [])
    event_type = headers.get("x-orbit-event-type", b"").decode("utf-8")
    handle(event, event_type)
```

Kafka records are one event per message — no array unpacking. The `x-orbit-event-type` header duplicates the envelope `type`, so a consumer that branches only on headers can skip parsing the value when you want to filter cheaply.

## Delivery semantics — batched, ordered, at-least-once

A sink delivery is **at-least-once**, same as a webhook:

* **Batched.** `http_batch` groups up to `max_batch_size` envelopes per POST; Kafka produces one record per event but `linger`-batches them inside the producer. Either way, retries that haven't yet exhausted their backoff can result in the same event appearing twice — dedupe on the envelope `id`.
* **Partition-ordered.** Kafka guarantees ordering only within a partition; that's why the partition key exists (a single contact's events are co-located). Ordering across two different contacts is not defined. `http_batch` delivers in flush order per tenant but a retry can interleave — rely on `created_at` for entity state, not receive order.
* **Retries.** A failed attempt (non-2xx on the HTTP POST, or a producer error on the Kafka send) is retried with exponential backoff, bounded at \~5s maximum delay, 3 attempts per record/batch. Each retry of the same event carries the identical serialized value — the bytes don't change mid-flight.

For the full retry/ordering/DLQ model as it applies to all surfaces, see [Webhook delivery semantics](/concepts/webhook-delivery-semantics).

## Inspect deliveries + DLQ mechanics

Both the dashboard page and a `GET` expose the sink's worker-owned run-status fields:

```bash theme={null}
curl "https://api.orbit.devotel.io/api/v1/developer/event-sinks/kafka" \
  -H "X-API-Key: dv_live_sk_your_key_here"
```

```json theme={null}
{
  "data": {
    "kind": "kafka",
    "enabled": true,
    "credentials_configured": true,
    "last_run_at": "2026-08-27T09:15:00Z",
    "last_run_status": "ok",
    "last_produced_count": 12,
    "last_error": null,
    "consecutive_failures": 0
  }
}
```

Read `last_run_status` as the health signal; `consecutive_failures` is a leading indicator even before you'd triage `last_error`. In the dashboard the same fields render as the status chip and health line on **Developer → Event Sinks**. When retries exhaust on a record or batch, the event is filed to a per-sink dead-letter list and the `last_run_status` flips to `failed` — the sink keeps consuming new events, it just stops retrying the poisoned one. You can run your own error-handling on your side based on the delivery outcome (your consumer logs + the sink's run status are the two surfaces).

## Event sink vs webhook — comparison

|                 | Webhook endpoint                                   | Event sink (Kafka or HTTP batch)                                      |
| --------------- | -------------------------------------------------- | --------------------------------------------------------------------- |
| Destination     | One HTTPS URL per endpoint                         | One Kafka topic, or one HTTPS collector URL                           |
| Body per POST   | One event                                          | HTTP batch: array of events; Kafka: one record per event              |
| Auth            | `X-Orbit-Signature` HMAC per request               | Bearer token header (http\_batch) — SASL on Kafka                     |
| Retry           | 1 initial + 9 retries, long backoff                | 1 initial + 2 retries, short backoff                                  |
| Ordering        | None guaranteed, but per-endpoint attempts bounded | Partition-level for Kafka (`partition_key_path`); none for HTTP batch |
| DLQ             | Per-endpoint, 7-day replayable via dashboard/API   | Per-sink dead-letter list; surfaced via `last_run_status: "failed"`   |
| Endpoint state  | Auto-disables after \~50 consecutive failures      | Sink keeps running; failing runs do not disable it                    |
| Secret rotation | `/rotation/initiate` lifecycle, grace header       | `PATCH` a new `credentials`; old value invalid after save             |

Pick the one that matches your consumer's natural shape. A customer receiving webhooks today can keep them while onboarding a Kafka sink in parallel — the event taxonomy, envelope shape, and filter vocabulary are identical.

## Production checklist

* [ ] Collector `url` (http\_batch) or `brokers` + `destination.topic` (kafka) set and enabled.
* [ ] `credentials_configured: true` on a read — the secret round-tripped.
* [ ] `events` filter set to the tightest prefix you actually consume (`message.*`, not `*`, unless you genuinely handle all).
* [ ] Receiver dedupes on the envelope `id` — at-least-once re-delivers under retry.
* [ ] A nightly job reads the sink (`GET /api/v1/developer/event-sinks/{kind}`) and alerts on `last_run_status: "failed"` or a growing `consecutive_failures`.
* [ ] SASL credentials live in a secrets manager, not in source control.

## Next steps

* [Event Sinks API](/api-reference/event-sinks) — full request/response field reference, credential handling
* [Webhook fan-out](/concepts/webhook-fan-out-and-event-sinks) — how one event fans out to webhooks, the events buffer, and sinks
* [Webhooks overview](/webhooks/overview) — the per-event taxonomy this sink consumes
* [Build a durable webhook consumer](/guides/webhook-consumer) — the webhook-side counterpart