Skip to main content

Run your first event sink deployment

The event sink reference covers the transport contract; this guide is the end-to-end first-deployment path — pick a sink type, wire the destination with its credential, validate the envelope against your consumer, confirm events land, and keep the sink healthy in operation. Webhooks and sink deliveries are both push-based — the difference is the destination shape, not the delivery model. Nothing here is hidden: every step runs against the same GET/PATCH surface the references document.

1. When a sink beats webhooks and SSE

Run these signals before you wire anything — the right transport is a consumer-shape decision, not a preference:
  • Webhooks fit when a third party expects a single-purpose HTTPS URL, or when each endpoint handles a narrow event list. Past a handful of endpoints, you are running a receiver farm with per-endpoint signing, per-endpoint retry state, and per-endpoint auto-disable logic.
  • Server-sent events (SSE) fit interactive dashboards that can tolerate a dropped event on reconnect. They do not fit delivery you cannot afford to miss.
  • An event sink fits when you already run Kafka, when you want one destination carrying many event types, or when delivery volume makes per-POST signing wasteful. One collector endpoint or one topic replaces the receiver farm.
If your webhook endpoints are saturating on message.* volume, or your queue consumers (an SQS poller, a Kinesis consumer) are starving because each event arrives as an individual webhook, consolidate onto a sink and point it at the queue front. Sink records are byte-identical to the webhook envelope, so one parser serves both — and you can run webhooks and a sink in parallel while onboarding.

2. Source vs sink — pick the right direction

Orbit has three adjacent event surfaces. Pick the outbound one here:
  • Event sources (inbound) — a Kafka source consumes your topic into Orbit, so your own domain events (orders, payments, CRM updates) drive platform behaviour. Configure it under Developer → Event Sources; see Configure event sources.
  • Event sinks (outbound) — stream the platform’s own event taxonomy out to a destination you run. That is this guide; the transport reference is Configure event sinks.
  • Connector downloads — the Zapier, Make, and n8n app definitions on Developer → Connectors are a different surface entirely: pre-built automations on top of webhooks, generated as per-tenant app-definition files. If you are deciding between writing your own sink consumer and importing a connector, see Connector downloads first — it can remove the need to run anything yourself.
The rest of this guide is the outbound sink path.

3. Wire your first sink

Both sink transports are addressed by kind and share one envelope: http_batch (an HTTPS collector you run, receiving JSON arrays) and kafka (Orbit produces native records to your brokers). AWS queue destinations — SQS, Kinesis — sit behind the HTTP-batch transport: stand up a small collector that puts the array onto the queue, and the sink delivers to it.

Option A — HTTP batch

  • destination.kind must equal the path segment — PATCH /…​/http_batch with a Kafka destination body is a 400.
  • The collector url must be public https — private and metadata hosts are rejected before the config can ever persist.
  • credentials becomes the Authorization: Bearer header on every batch POST — encrypted at rest, write-only, and a read proves it stored via credentials_configured: true without ever returning the value.
  • Keep the events filter tight. message.* over * is the difference between a sink that stays cheap and one that carries every contact.* update you never consume.
For an SQS front, the collector is a few lines — receive the array, verify the Bearer token you set, put each entry on the queue, and 200 only when the batch is durably enqueued:
The same fields are settable in the dashboard under Developer → Event Sinks.

Option B — Kafka

Brokers are host:port entries with no scheme or path; the topic name follows Kafka naming rules (1–249 chars, [A-Za-z0-9._-], never . or ..). If your brokers are not SASL-protected, omit the three SASL fields entirely — the credentials here carries the SASL password plaintext in the request, encrypted at rest once stored.

Validate the envelope against your consumer

Every event is serialized to the same shape the Webhooks overview documents — id, type, created_at, and a data payload:
Write your consumer against that shape now, before you ever receive a delivery: branch on type, dedupe on the envelope id (the delivery worker produces at-least-once — retries re-deliver byte-identical records), and treat created_at as the entity-state clock rather than receive order. On http_batch the body is a JSON array of these envelopes (up to max_batch_size per POST); on Kafka each record is one envelope, with the partition key resolved from your partition_key_path and a flattened set of headers (x-orbit-event-type plus any custom headers you configured).

Send a test event

Generate real traffic the cheapest way you can (send yourself a test SMS, complete a test call), then read the sink back:
credentials_configured: true proves the secret round-tripped and last_produced_count climbing past zero proves events are landing — seeing last_run_status: "ok" on a read is the green light to cut your consumer over. In the dashboard, flip a webhook subscription in parallel while you onboard the sink so you can compare delivery across both surfaces; once your filter matches and you see an envelope on the sink, disable the webhook endpoint and hand production over to it.

4. Operate

The sink’s own run-status fields are the health signal, and delivery is at-least-once — failure is never silent; it surfaces here:
  • last_run_status: "ok" — the last delivery attempt succeeded. "failed" means retries exhausted on the last record or batch.
  • consecutive_failures — a rising count is the leading indicator even before you triage last_error. Alert on anything above zero.
  • last_produced_count — how many events the last run delivered; a sudden drop with enabled: true usually means your events filter tightened past what you actually consume.
Retry and backpressure, then the dead letter:
  • Retries. A failed attempt (non-2xx on the HTTP POST, or a Kafka producer error) is retried with exponential backoff bounded at ~5s, 3 attempts per record or batch. Each retry carries the identical serialized bytes — which is why deduping on id works.
  • Dead letters. When retries exhaust, the event files to the sink’s dead-letter list and last_run_status flips to "failed". The sink keeps consuming new events; only the poisoned one stops.
  • Backpressure at your receiver. The sink reads any non-2xx as a retry signal. If your collector is saturated, answer with a 429 (or 5xx) rather than queuing unsustainably, and drain at your own pace; the bounded backoff spreads the redeliveries. If you need a stronger lever, lower max_batch_size so each POST is a smaller unit of work.
  • Replay belongs to your side of the line: on http_batch, re-POST the envelope to your collector from your own dead-letter tooling; on Kafka, the record is already on your brokers, so re-run your consumer with the same id dedupes correctly. Your consumer logs plus the sink’s run status are the two surfaces of truth. For webhook-surface dead letters, the dashboard offers a 7-day replayable DLQ.
Backpressure to watch:
  • http_batch — your collector must ack the whole batch with 2xx. A non-2xx retries the batch; sustained retries with a growing JSON array will eventually hit your own endpoint’s caps.
  • Kafka — ordering is partition-scoped, which is why the partition key path exists. Ordering across two different contacts is not defined; rely on created_at for entity state, not receive order.
Run a nightly job that reads GET /api/v1/developer/event-sinks/:kind and alerts on last_run_status: "failed" or a rising consecutive_failures. That is the whole monitoring contract — one field tells you the sink is healthy.

5. Scale

Two sinks are independent by design — one Kafka, one HTTP batch, either, both, or neither. Fan-out is a configuration decision, not a pipeline problem:
  • Split by event family — route message.* to your queue front (http_batch → SQS) and call.* to Kafka, so each consumer pool scales against its own traffic shape.
  • Warehouse fan-out — if the end goal is analytics rather than real-time consumption, mirror contacts and event streams into Snowflake, BigQuery, or another warehouse on a cadence instead of running a receiver at all. That path is CDP reverse ETL to Snowflake and BigQuery — the /api/v1/cdp/reverse-etl surface that mirrors CDP profiles and events into Snowflake, BigQuery, Redshift, self-managed Postgres, Databricks, or ClickHouse on an operator-set cadence. The sink’s partition_key_path (co-locating one contact’s events) is what makes per-contact ordering hold downstream.
  • Both directions at once — keep your existing webhook endpoints live while you onboard the sink; the event taxonomy, envelope shape, and filter vocabulary are identical, so cutover is a consumer change, not a data change.

Production checklist

  • Destination set (http_batch collector URL, or Kafka brokers + destination.topic) and enabled: true.
  • credentials_configured: true on a read — the secret round-tripped.
  • events filter at the tightest prefix you actually consume (message.*, not *, unless you genuinely handle all).
  • Receiver dedupes on the envelope id — at-least-once re-delivers under retry.
  • Collector answers 429/5xx under saturation instead of queuing unboundedly.
  • A nightly reader alerts on last_run_status: "failed" or growing consecutive_failures.
  • Secrets live in a secrets manager, not in source control.
  • Webhook endpoint(s) disabled once the sink has verified your filter on the live stream.

Next steps