> ## Documentation Index
> Fetch the complete documentation index at: https://docs.orbit.devotel.io/llms.txt
> Use this file to discover all available pages before exploring further.

# Troubleshooting: duplicate webhook events and consumer-side dedup

> Handle duplicate webhook deliveries — one event arriving more than once, bulk replay re-sending succeeded events, and event-id collisions on your consumer — by keying dedup on the stable event id and acking fast.

# Troubleshooting: duplicate webhook events and consumer-side dedup

Your ledger shows two credits for one delivered message, a bulk replay fired
ten thousand events your consumer already processed, or your handler rejects
an event it has genuinely never seen. Almost every "webhook sends duplicates"
report lands in one of three buckets: Orbit retried because it could not see
your ack, you replayed deliberately and the consumer could not recognize the
re-delivery, or the consumer is hashing the wrong key. This page maps each
symptom, then gives you the consumer pattern that makes all three safe.

Failure diagnosis — your endpoint returning 5xx, timing out, or deliveries
landing in the dead-letter queue — is a different loop; start with
[Troubleshooting: failed webhook deliveries, retries, DLQ, and replay](/troubleshooting/webhook-deliveries)
and come back here once deliveries flow.

## Symptom map

Match the duplicate you see to its cause before changing any code:

| Symptom                                                                       | What it usually means                                                                                     | Where to confirm                                                                                               |
| ----------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------- |
| Same `message.delivered` fires twice, seconds apart                           | An "ambiguous outcome": your endpoint acked, but Orbit timed out waiting for it and retried at-least-once | `GET /api/v1/webhooks/{id}/deliveries` — two delivery rows sharing one `event_id`, the first showing a timeout |
| Events arrive double after `POST /api/v1/webhooks/{id}/replay-range`          | Bulk replay re-sends every stored delivery in the window, **including the ones that succeeded**           | The replay job's window vs. deliveries that already show `success`                                             |
| Concurrent `POST` on one event, handler crashes on a lock or uniqueness error | Your handler treats the delivery itself as the identity instead of the event id inside it                 | Your consumer logs — two simultaneous inserts with the same `id` (`evt_...`)                                   |
| Your dedup table grows forever / rejects a genuinely new event                | You are keying on the payload body or the delivery id instead of the event id                             | Compare the key you persist against `body.id` and `Idempotency-Key`                                            |

Each row expands below.

## How Orbit emits a duplicate: the dedup model

Orbit's dispatch pipeline is **at-least-once by design**. One event — one
`evt_...` id — is persisted once in the webhook-events registry, then
attempted against your endpoint up to 10 times (initial delivery plus 9
retries over roughly 4.3 hours) until an ack lands or the row reaches the
DLQ. Two of those attempts produce a delivery you must treat as a duplicate:

* **Duplicate on retry.** A retry is only started when Orbit did not see a
  `2xx` in time. If your handler did its side effect and then timed out —
  or returned the ack after Orbit's per-endpoint `timeout_seconds` (30-second
  default) — the retry you receive is a duplicate of an event you already
  processed.
* **Duplicate on replay.** Replay re-dispatches the stored payload through
  the same pipeline, by hand. It works on succeeded rows too: that is the
  feature, and it always re-sends the *same* event id.

The dedup anchor is stable across both paths and lives in two places:

* The **envelope `id`** — `"id": "evt_..."` in the JSON body.
* The **`Idempotency-Key` header** — mirrors `body.id`, so you can dedup
  before parsing the body.

Both stay identical across every retry and every replay of one event. The
header shape is documented on
[Webhook event payloads](/webhooks/event-payloads); the dispatch/retry
semantics behind them are on [Webhook security](/webhooks/security).

## The durable-consumer pattern

Orbit's guarantee ends at "the same event id may arrive more than once."
The idempotency guarantee on your side is tenant-owned: ack fast, process
through a queue, and persist the event id before you ack.

1. **Verify the signature**, then read `body.id` (or the `Idempotency-Key`
   header — same string).
2. **Persist the id** into a table or key store with a unique constraint,
   keeping it for at least 7 days (the full retry window plus the DLQ
   retention window).
3. **Ack immediately.** If the insert hit the unique constraint, the event
   is a duplicate: still return `200`. A fast duplicate ack stops the retry
   cadence and does not re-fire your billable work.
4. **Process through a queue.** Push the event onto your own worker queue
   and let the worker, not the HTTP handler, run the side effect. If the
   worker crashes, the event retries on your side — Orbit never has to.

```ts theme={null}
// In your POST handler — ack in well under 30 s, every time
async function onOrbitWebhook(req, res) {
  verifySignature(req);                       // 401 + stop on failure
  const eventId = req.body.id;                // "evt_..." — or header Idempotency-Key
  const inserted = await seen.insertIgnore(eventId);  // unique ON body.id
  if (!inserted) return res.status(200).json({ duplicate: true });
  await queue.enqueue(req.body);              // side effects belong to a worker
  return res.status(200).json({ received: true });
}
```

The full build, with a worked Node and Python receiver, is in
[Build a durable webhook consumer](/guides/webhook-consumer). Two
consequences fall out of the pattern:

* **The only correct key is the event id** — never the delivery id (each
  attempt has its own), never a hash of the payload body (the same logical
  event can legitimately be replayed with identical bytes, and identical
  bytes keyed wrong still collapse into one row).
* **Ack the duplicate, don't 4xx it.** Returning anything but `2xx`
  burns retry budget on an event you already processed and can push a
  healthy endpoint toward auto-disable (50 consecutive failures).

## Replay-side dedup

Replay re-sends events that may already have succeeded, so "what to dedup"
depends on which replay you ran:

* **DLQ requeue is safe by construction.** `POST /api/v1/webhooks/dlq/{deliveryId}/requeue`
  only moves rows that *never* succeeded back into the retry scheduler —
  your consumer has not seen those event ids yet.
* **Single-delivery replay** — `POST /api/v1/webhooks/{id}/deliveries/{deliveryId}/replay`
  — re-sends one stored event. If the original attempt succeeded, the
  consumer's event-id insert collapses it to a no-op; coverage is not lost
  because the key is the same.
* **Bulk replay over a window** — `POST /api/v1/webhooks/{id}/replay-range`
  — is the dedup-critical case. The window covers *every* stored delivery,
  not just failed ones, so the burst contains a share of duplicates by
  definition. Keep the window tight around the outage, scope it with
  `eventTypeFilter` where the outage was event-specific, and let the
  durable-consumer pattern above absorb the share that already succeeded.
  Replaying a wide window to a non-durable consumer is the one case that
  turns a recovery tool into a double-processing storm.

The full requeue-versus-replay decision, the one-job-per-endpoint `409`
constraint, and job polling are on
[Troubleshooting: failed webhook deliveries, retries, DLQ, and replay](/troubleshooting/webhook-deliveries).

## Case studies

### Case 1 — duplicate `message.delivered`

**Symptom.** A customer's SMS ledger posts two credits for one delivered
message. The dashboard duplicates page shows two delivery rows for the same
endpoint, 30 seconds apart.

**Confirm.** Pull the deliveries for the endpoint:

```bash theme={null}
curl -G "https://api.orbit.devotel.io/api/v1/webhooks/{id}/deliveries" \
  -H "X-API-Key: dv_live_sk_..." \
  --data-urlencode "event_type=message.delivered"
```

You see two rows that share one `event_id` (`evt_...`) but carry different
delivery ids. The first row's attempt ends in a timeout or a non-2xx; the
second succeeds. Orbit retried because it never saw the ack — this is the
delivered-as-designed duplicate your consumer is expected to absorb.

**Fix.** The handler was posting the credit synchronously and acking after
the database write. Move the credit into a queue job, persist `body.id`
before acking, and return `200` as soon as the id is durably stored. The
next retry of that event id is no longer a second credit — it is an ack
from your dedup store.

### Case 2 — duplicate `conference.participant_joined`

**Symptom.** After a receiver outage, you bulk-replayed a three-hour
window. Every participant row doubled — the "joined" count is now twice
the real attendance.

**Confirm.** Fetch the replay job
(`GET /api/v1/webhooks/{id}/replay-range/{jobId}`) and compare its
`from`/`to` against the deliveries list. The window you replayed contains
deliveries that had already succeeded before the outage, so each event in
that share arrived twice — with the same `evt_...` id both times.

**Fix.** The consumer was keyed on `(conference_id, participant_session_id)`
derived from the payload, not on the event id, so nothing it persisted
could recognize the replay. Switch the dedup key to `body.id` with the
7-day retention from the durable-consumer pattern. Re-run a tight replay
window after the fix — succeeded rows no-op, the rows that had genuinely
never landed recover, and the count converges.

## What to send support

If you are seeing duplicate deliveries whose delivery rows do **not** share
an event id (the same event genuinely minted twice), or duplicates persist
after your consumer acks in under a second, write in with:

* **Event id** (`evt_...`) from the duplicate pair — the single strongest
  signal.
* **Delivery ids** (`wdl_...` / the delivery row ids) of both attempts,
  from the deliveries list.
* **Consumer-side timestamps** — when your endpoint received each attempt
  and which status code it returned, in UTC.
* **Webhook endpoint id** and **tenant id** (Settings → Organization).
* For replay-caused duplicates, the **job id** and the `from`/`to` window
  you replayed.

That set lets support line up our dispatch log against your access log in
one pass instead of three.

## See also

* [Troubleshooting: failed webhook deliveries, retries, DLQ, and replay](/troubleshooting/webhook-deliveries) —
  the failure-and-recovery loop this page deliberately skips.
* [Build a durable webhook consumer](/guides/webhook-consumer) — the
  worked receiver that implements the pattern above.
* [Webhook event payloads](/webhooks/event-payloads) — the envelope and
  the `Idempotency-Key` header contract.
* [Webhook security](/webhooks/security) — delivery, retry, and signature
  semantics.
* [Webhook deliveries: explore, inspect, replay](/webhooks/inspecting-deliveries) —
  the dashboard surfaces for reading attempts.