> ## Documentation Index
> Fetch the complete documentation index at: https://docs.orbit.devotel.io/llms.txt
> Use this file to discover all available pages before exploring further.

# Troubleshooting: failed webhook deliveries, retries, DLQ, and replay

> Read failed webhook deliveries end to end — decode the retry cadence, find deliveries in the dead-letter queue, and choose between requeue, single-delivery replay, and bulk replay without double-processing.

# Troubleshooting: failed webhook deliveries, retries, DLQ, and replay

Your dashboard shows failed deliveries, your endpoint returned 5xx or timed
out, or events simply stopped arriving. This page walks you through the
delivery lifecycle, from the first failure to a deliberate replay, and tells
you which recovery tool fits which failure.

Signature failures are a different class — if your endpoint rejects every
delivery with a 401 or a signature error, fix verification first with
[Troubleshooting: signature failures](/webhooks/troubleshooting-signature-failures)
and come back here once the endpoint accepts requests.

## Symptom map

Match what you see to the underlying cause before reaching for a recovery
tool:

| Symptom                             | What it usually means                                                        | Where to confirm                                                                           |
| ----------------------------------- | ---------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------ |
| Dashboard shows `failed` deliveries | Your endpoint returned non-2xx or timed out on those attempts                | **Developer → Webhooks → Events**, or `GET /api/v1/webhooks/{id}/deliveries?status=failed` |
| Your endpoint returns 5xx           | Your consumer or its dependency is down; Orbit retries automatically         | Watch the retry cadence below; the row lands in the DLQ after exhaustion                   |
| Your endpoint times out             | The handler is too slow — ack fast and hand work to a queue                  | Compare against your endpoint's `timeout_seconds` setting                                  |
| Events stop arriving entirely       | The endpoint is paused, unregistered, or rejects everything (signature/SSRF) | `GET /api/v1/webhooks/{id}` shows `paused`; the health endpoint shows the recent outcomes  |
| Rows sit in the dead-letter queue   | All nine retry attempts ran and none succeeded                               | `GET /api/v1/webhooks/dlq`                                                                 |

Also check the reliability roll-up (`GET /api/v1/webhooks/reliability`) when
you cannot tell whether one endpoint or the whole tenant is affected — it
summarizes success rate and the oldest pending delivery per endpoint over a
24-hour window.

## Retry cadence: when a row lands in the DLQ

Orbit retries a failed delivery automatically: up to **9 retries** on an
exponential backoff with a 30-second base delay that doubles with each
attempt, plus up to 20% jitter — roughly 4.3 hours from the first failure to
the final attempt, per the
[FAQ](/reference/faq) and the
[Webhook security](/webhooks/security) page's delivery contract.

Two implications:

* **Do not manually requeue a row that is still retrying.** A row that shows
  `failed` on an attempt is not exhausted until it appears in the DLQ. Wait
  for the scheduler to finish its 4.3-hour window before treating the event
  as lost.
* **Fix the endpoint before the scheduler finishes.** If a deploy broke the
  consumer, a hotfix redeployed within the window lets one of the remaining
  retries succeed on its own — no manual action needed.

After the final attempt fails, the delivery moves to the dead-letter queue
with status `dlq` and remains retrievable.

## Step 1 — Read the delivery attempts

List the attempts for the endpoint and filter to the failure:

```bash theme={null}
curl -G "https://api.orbit.devotel.io/api/v1/webhooks/{id}/deliveries" \
  -H "X-API-Key: dv_live_sk_..." \
  --data-urlencode "status=failed"
```

Cursor-paginate through the window you care about (`from_date` / `to_date`
bounds it) and pull the individual delivery
(`GET /api/v1/webhooks/{id}/deliveries/{deliveryId}`) to read the exact
payload Orbit sent and the HTTP status your endpoint returned. That pair
answers *what failed*: a 404 means the route moved, a 502/503 means the app
crashed or its upstream did, a timeout means the handler ran past the
endpoint's `timeout_seconds`. For the request shape and filters, see
[List webhook deliveries](/api-reference/endpoints/webhooks).

## Step 2 — DLQ listing and single-delivery requeue

List the dead-letter queue, optionally scoped to one endpoint:

```bash theme={null}
curl -G "https://api.orbit.devotel.io/api/v1/webhooks/dlq" \
  -H "X-API-Key: dv_live_sk_..." \
  --data-urlencode "endpoint_id={id}"
```

**When to requeue a single delivery** — with
`POST /api/v1/webhooks/dlq/{deliveryId}/requeue`: one row, still meaningful,
after you confirmed the endpoint is healthy again. Requeueing resets the
delivery to `pending` and clears its attempt counter, so the retry scheduler
picks it up with a fresh nine-attempt budget.

**When NOT to requeue** — when the endpoint is still failing. A requeue
into an endpoint that still returns 5xx just burns the fresh budget and puts
the row back in the DLQ nine attempts later. Fix the consumer first, then
requeue. If hundreds of rows are queued behind the same outage, a single
requeue per row is the wrong tool — that is what bulk replay is for.

## Step 3 — Replay: single delivery and bulk

Requeue and replay are not interchangeable:

* **Requeue** moves a DLQ row back into automatic retries. Use it when the
  failure was transient and you want the normal scheduler to handle it.
* **Replay** re-dispatches the stored payload through the standard dispatch
  pipeline, on demand. Unlike requeue, replay works on succeeded deliveries
  too — which is exactly why it can double-process (see idempotency below).

**Single-delivery replay** — `POST /api/v1/webhooks/{id}/deliveries/{deliveryId}/replay` —
when you need that one event re-sent now rather than on the retry cadence:
the customer is on the phone, or the event gates a downstream workflow.

**Bulk replay** — `POST /api/v1/webhooks/{id}/replay-range` — when an outage
caused a window of failures and the DLQ rows are all attributable to it. The
request takes a `from`/`to` window and an optional `eventTypeFilter`, returns
`202` with a `job_id`, and runs as a background job. Poll it with
`GET /api/v1/webhooks/{id}/replay-range/{jobId}` until it reports `done` or
`failed`:

```bash theme={null}
curl -X POST "https://api.orbit.devotel.io/api/v1/webhooks/{id}/replay-range" \
  -H "X-API-Key: dv_live_sk_..." \
  -H "Content-Type: application/json" \
  -d '{
    "from": "2026-08-20T09:00:00Z",
    "to": "2026-08-20T12:30:00Z",
    "eventTypeFilter": ["message.delivered"]
  }'
```

```bash theme={null}
curl "https://api.orbit.devotel.io/api/v1/webhooks/{id}/replay-range/{jobId}" \
  -H "X-API-Key: dv_live_sk_..."
```

Constraints to plan around:

* **One bulk replay per endpoint.** A second start while a job is queued or
  running returns `409` with error code `BULK_REPLAY_IN_PROGRESS`. Wait for
  the in-flight job to finish rather than looping on the 409.
* **Keep the window tight.** Replay re-sends *every* stored delivery in the
  window, including ones that succeeded. A window wider than the outage
  replays events your consumer already processed — see idempotency below.

**When not to replay a row**: when your consumer still is not idempotent and
the row may already have been processed (a succeeded row in the window, or a
row on which the handler reached its side effect before the ack failed).
Fix the consumer's dedup first — the next section — then replay.

## Idempotency: replay-safe consumers

Replay re-delivers the original payload with the same event id. A consumer
that already persisted the event must recognize the re-delivery and no-op,
because replay deliberately re-sends events that may have succeeded. The
durable-consumer pattern — persist the seen `event id`, ack fast, process
through a queue — makes every recovery tool above safe to use. Build that
first, then replay is a routine operation instead of a double-processing
risk: see the
[Build a durable webhook consumer](/guides/webhook-consumer) guide.

## What to send support

If deliveries keep failing after the endpoint returns 2xx consistently, or a
bulk replay job reports `failed`, collect the identifiers before you write
in — each one removes a round-trip:

* **Tenant id** (Settings → Organization)
* **Webhook endpoint id** (the endpoint from the dashboard URL or `GET /api/v1/webhooks`)
* **Delivery id from the failed row** (from the deliveries list or DLQ)
* For bulk replay problems, the **job id** from the `202` response, plus the
  `from`/`to` window you replayed
* The **time range** of the failures you saw, in UTC

## See also

* [Webhook deliveries: explore, inspect, replay](/webhooks/inspecting-deliveries) —
  the dashboard surfaces for the same loop.
* [Troubleshooting: signature failures](/webhooks/troubleshooting-signature-failures) —
  when the endpoint rejects every request, fix HMAC verification first.
* [Build a durable webhook consumer](/guides/webhook-consumer) — the
  idempotency pattern that makes replay safe.
* [Webhook security](/webhooks/security) — the delivery contract, signature,
  and retry semantics.