Skip to main content

The export families model

Several surfaces in Orbit move data out of your workspace, and the decision they answer is different: ad-hoc audience exports, conversation exports, CDP reverse-ETL pushes, WORM archival bundles — and, beside the export families, the streaming event sinks. Picking the wrong one wastes a time-bounded bundle, holds up a compliance process, or stands up a warehouse you never query. This page frames the decision.

1. The export families compared — continuous vs on-demand vs evidentiary

The right question is not “which format” but “which category of egress”:
  • Continuous replication — the data keeps flowing on a schedule or a stream, so your downstream copy stays current. Reverse-ETL destinations (batch schedules) and event sinks (near-real-time event streams) live here.
  • On-demand artifacts — you ask for a point-in-time artifact: a filtered audience CSV, or a signed vCon of one conversation. Ad-hoc CSV and the conversation vCon live here.
  • Evidentiary archives — you need records you can later prove were never altered: WORM archival bundles with a signed manifest and hash chain.

2. What each format guarantees

  • Ad-hoc CSV is point-in-time and non-restartable. The filter resolves against your live contacts the moment you call it, up to a 50,000-row ceiling (500,000 on the saved-export path). The X-Export-Truncated header tells you whether the window cut off. Re-run the request to re-export after membership changes; nothing persists between runs. Per-record PII masking matches the caller’s reveal context.
  • vCon captures conversation state. Parties, dialog, attachments, and analysis sections, signed at export time so the recipient can prove it was not altered. The container mints a fresh uuid per export rather than exposing the internal conversation id.
  • Reverse-ETL is a scheduled push into the destination profile catalog. Section 3 below breaks the model down.
  • WORM is tamper-evident signing plus a hash chain. Every export returns a removable bundle containing a signed manifest, a per-file SHA-256 checksum table, and verification instructions. Optionally encrypted under a one-time passphrase Orbit never stores. The lock promise (window, basis, version) is stamped into the export manifest so an auditor can check the policy was still in force when records were captured.

3. The reverse-ETL destination model, deeper

The reverse-ETL lane is the second-largest of the families and the one with a per-destination model worth understanding before you configure anything. Three concepts carry it: the destination profile catalog, the trait-resolution verdict per warehouse, and the per-destination run-status ledger.

The destination profile catalog

The shipped catalog is six destinations: BigQuery, Snowflake, Redshift, Postgres, Databricks, and ClickHouse. You configure and enable each under your CDP destinations panel (GET /api/v1/cdp/reverse-etl/destinations aggregates all six into one panel model). Each destination holds its connection details (host or account, database, schema, cadence) plus a write-only credential — BigQuery service-account JSON, Snowflake key pair, Redshift/Postgres connection credentials. The credential is stored encrypted and never returns from any read surface; reads expose a configured/not-configured badge, not the secret. That credential shape is exactly why these warehouses are destination-shaped, not sink-shaped: a destination is a named system that accepts full-table pushes on a schedule, guarded by a durable credential your data team owns and rotates. A sink is a transport that accepts individual events as they happen. The warehouse’s security model — one long-lived credential that authorizes table writes — matches the destination shape; the event plane’s per-envelope model matches the sink shape (the event-sinks streaming guide covers that plane).

Profile-trait resolution per warehouse: native vs organic vs not_configured

Not every destination treats your computed profile traits the same way downstream. The read model at GET /api/v1/cdp/reverse-etl/profile-exports joins your trait catalog against destination config and classifies each destination into exactly one verdict: Per trait, that verdict composes with the trait’s own status into a propagation decision: syncable: true needs a native verdict on an enabled destination and an active trait; anything else reports syncable: false with an explicit block_reason (destination_not_configured, destination_disabled, traits_via_exported_data, trait_not_active). A paused trait shows up with its reason rather than vanishing from the panel.

The per-destination run-status ledger

Every run — success or failure — writes a snapshot back onto that destination’s config state, and GET /api/v1/cdp/reverse-etl/sync-runs projects the latest ledger entry for all six destinations in one response. Per destination:
  • last_run_at — when the most recent run ran.
  • last_run_status — ok or failed.
  • last_run_rows_contacts / last_run_rows_events (with rows_total as the convenience sum) — exported row counts per table.
  • last_error + last_error_at, consecutive_failures — the failure detail.
  • health — a derived, most-actionable-first verdict: disabled → never_run → failing → stale → ok.
The two loud verdicts keep a broken export from failing silent: failing means the most recent run returned failed and the row carries the error; stale means the last successful run is older than the expected cadence (48h against the default nightly schedule), so “the scheduler stopped firing” surfaces in one query.

4. The archive vs event-sink split

Reverse-ETL destinations are scheduled batch: they tolerate single-night staleness by design and answer “is my warehouse copy current?” The event-sink plane is streaming: one declared destination (kafka or http_batch) receives events as they happen, with batching, retry, and delivery health of its own. They complement each other:
  • Use reverse-ETL when the consumer is your warehouse and its BI/ML/finance queries — trait columns, profile aggregates, and historical joins.
  • Use an event sink when the consumer reacts to events — alerting, orchestration, feature pipelines that need minutes-level freshness.
  • Neither proves a record was never altered; the WORM archival layer answers that question with signed bundles and a hash chain.

5. Common denominators

Every export family shares a core safety model, regardless of lane:
  • Tenant-scoped. Every export runs against documents in your own tenant schema; a cross-tenant export is not a thing the API can do.
  • Per-record ownership checks. The request authorizes as the workspace it claims to belong to; a record outside that workspace never enters the export.
  • Time-bounded URIs. vCon responses, archival download URLs, and streaming export payloads generated to a signed URL are time-bounded — pick up the artifact while the URL is valid, then mirror it into your own store. Re-run the export to revive an expired URL.
  • Signed URLs where applicable. Archival exports are the canonical example: a signed URL (7-day expiry on archival bundles) plus the checksum block let your mirroring job prove the artifact it fetched matches what Orbit produced.
  • Destination credentials are write-only. On the reverse-ETL lanes, the credential encrypts on save and never returns from any read surface — rotates when your data team rotates, without leaking into dashboards or exports.

6. Failure semantics

  • WORM hash mismatch. Your mirroring flow treats the archive as evidence to verify, not just fetch. The export response returns bundleSha256 and a manifest signature; the bundled VERIFY.txt documents how to replay the per-file checksums in your own environment. On audit-log exports, GET /api/v1/compliance/audit-export/:jobId/verify replays the hash chain and returns a replay-status verdict (prev_hash_mismatch or current_hash_mismatch flag the broken link). Either way, a mismatch is loud rather than silent — the bundle never lands in your archive.
  • Reverse-ETL retries. Scheduled pushes persist the per-destination run-status ledger above — failing destinations surface the last error rather than fail silent, and stale flags runs older than the expected cadence. A failed “sync now” keeps the request set so the next tick retries.

7. Decision table

8. Where the per-family guides live

Ad-hoc audience export

One-shot CSV pulls from a live filter, the saved-segment export trade-off, and when a segment is the right home.

Conversation vCon

The signed IETF container, masking preferences, and audit-log coverage of every export.

Reverse-ETL destination model

The per-destination verdicts, schedule semantics, and run-status ledger — the concept page this family’s guide backs.

Reverse-ETL warehouse exports guide

Configure each warehouse destination, set the cadence, read per-trait propagation verdicts, show sync-now.

Event sinks streaming guide

The continuous-streaming alternative when minutes-level freshness beats the warehouse cadence.

WORM archival export

Policy, signed bundles, retention-lock semantics, and mirroring into your own WORM or S3 store.

Retention windows and deletion

The cross-store lifecycle map: what each data class lives for by default, and how WORM archival replaces deletion.