Troubleshooting: voice biometrics failures

The enroll and verify calls on /api/v1/verify/voice-biometrics hand each audio sample to the inference sidecar, and every failure this page covers is one of the ways that hand-off can break. The result vocabulary on a 200 (passed, failed_no_match, failed_spoof, no_enrollment) is the decision layer — covered on Voice biometrics, not here. This page owns the layer underneath it: the calls that never got a decision because the sidecar did not answer, rejected the signature, or answered with something unusable. The handful of codes below are deliberately few. Read error.code off the envelope and you know which side owns the fix — connectivity codes are yours to retry, the auth code means a secret drift to resolve, and the config code is an operator-side fix on our deployment.

Code matrix — symptom to fix

Error code	HTTP	Endpoint it fires on	What it means	First move
`VOICE_BIOMETRICS_TIMEOUT`	503	`POST /enroll`, `POST /verify`	The sidecar did not answer within the 10-second inference budget	Retry once with backoff — see Sidecar connectivity
`VOICE_BIOMETRICS_UNAVAILABLE`	503	`POST /enroll`, `POST /verify`, `POST /challenge`	The sidecar (or the challenge store) was unreachable — a network-layer failure, not a slow response	Retry with backoff; persistent failures across minutes are a platform condition — see Sidecar connectivity
`VOICE_BIOMETRICS_AUTH_FAILED`	500	`POST /enroll`, `POST /verify`	The internal HMAC signature between the API and the sidecar did not verify	Do not retry in a loop — the secret pair drifted and a retry signs identically. See Sidecar auth
`VOICE_BIOMETRICS_ERROR`	503	`POST /enroll`, `POST /verify`	The sidecar returned an unexpected HTTP status, a non-JSON body, or a body missing required fields	Capture and escalate — see Unexpected sidecar error
`CONFIGURATION_ERROR`	500	`POST /enroll`, `POST /verify`	The internal sidecar-signing secret is missing from the deployment	Nothing tenant-side fixes this — see Configuration

Codes that share an endpoint but are not sidecar failures — INVALID_AUDIO (undecodable base64, or a rejected sample), AUDIO_TOO_LARGE (over the 30-second / 2 MB sample cap), CHALLENGE_INVALID (a half-passed, expired, or mismatched liveness nonce), SPOOF_DETECTED, DEEPFAKE_DETECTED, and DUPLICATE_VOICEPRINT — are deterministic request- or biometric-layer verdicts: fix the audio or the flow, not the connectivity.

Sidecar connectivity

Both codes here are 503s, and both mean the audio never reached a working analyzer. The distinction matters for what you tell us:

VOICE_BIOMETRICS_TIMEOUT — the sidecar connection opened but no response arrived within 10 seconds. Normal inference completes well under a second, so a timeout points at an overloaded or wedged sidecar, not a slow link.
VOICE_BIOMETRICS_UNAVAILABLE — the request never got as far as a response: connection refused, DNS, or a reset. Enroll and verify raise this against the sidecar; POST /challenge raises the same code against the challenge store when it cannot persist a nonce.

Cause table

Symptom	Likely cause	Fix
A burst of 503s during one window, then clean	Sidecar restart or rollout	Retry after a short backoff; the failure window should close on its own
Timeouts only, on longer samples	Sample near the 30-second cap hitting the inference budget	Trim the sample to the spoken phrase; large samples spend the budget
Unavailable on `POST /challenge` while enroll/verify stay green	The challenge store, not the sidecar, is down	Either retry later, or send verify without challenge fields and keep the liveness gate off for that call
Persistent timeouts or unavailability across minutes, both endpoints	Platform-side sidecar condition	Retry with backoff for a bounded window, then escalate with the samples below

Retry-safety

Both 503 codes are safe to retry. Enroll and verify do not write anything before the sidecar answers — a retry after a connectivity 503 re-runs the same attempt end to end, it cannot double-enroll a voiceprint (a genuine duplicate sample is rejected with DUPLICATE_VOICEPRINT 409, a different envelope entirely). Use exponential backoff rather than a tight loop: three retries spaced over tens of seconds covers a rolling restart; a millisecond-interval loop spends your rate limit on the same outage. On POST /challenge, prefer re-issuing a challenge after a failure over reusing one — a consumed or partial nonce is the one state a retry cannot repair.

Envelope sample

{
  "error": {
    "code": "VOICE_BIOMETRICS_TIMEOUT",
    "message": "Voice biometrics sidecar did not respond within 10 seconds. Check that voice-biometrics-svc pods are healthy and models are loaded.",
    "status": 503
  },
  "meta": {
    "request_id": "req_vbs_tmout1",
    "timestamp": "2026-09-12T09:41:07Z"
  }
}

Ticket checklist

If retries over a bounded window keep returning the same 503, open a ticket with:

The code verbatim (VOICE_BIOMETRICS_TIMEOUT or VOICE_BIOMETRICS_UNAVAILABLE) and the endpoint you called.
The request IDs off meta.request_id for two or three of the failing calls, with their timestamps — that brackets the outage window.
Your organization ID (Settings → Organization, or organizationId from GET /api/v1/me).
Whether enroll and verify both fail, or only one — the split separates a sidecar-wide condition from a single-path one.

Sidecar auth

VOICE_BIOMETRICS_AUTH_FAILED (500) fires on one thing only: the sidecar rejected the internal HMAC signature on the request. The public half of the call already authenticated — your API key was accepted, the route ran, and the failure happened on the internal hop to the analyzer. Your integration cannot cause this, and nothing you change in the request retires it. The signature is computed per request from a shared secret pair held on the API and the sidecar. This error means the pair drifted — almost always a secret rotation that landed on one side and not the other, or a stale sidecar still signing against the previous value.

Secret rotation checklist

Do not retry in a loop. Every retry signs with the same drifted secret and fails identically — a retry storm adds load without changing the outcome.
Check the timestamp spread. A healthy-to-failing cutover at one instant marks the rotation moment; interleaved successes and failures mark a half-rolled deployment.
Escalate immediately. Pair realignment is operator-side. Include the first failing request ID and the last succeeding one — that narrows the rotation window to minutes.

Envelope sample

{
  "error": {
    "code": "VOICE_BIOMETRICS_AUTH_FAILED",
    "message": "HMAC authentication with voice biometrics sidecar failed. Check DEVOTEL_VOICE_BIOMETRICS_INTERNAL_SECRET matches between API + sidecar.",
    "status": 500
  },
  "meta": {
    "request_id": "req_vbs_auth1",
    "timestamp": "2026-09-12T09:55:12Z"
  }
}

Ticket checklist

The code (VOICE_BIOMETRICS_AUTH_FAILED) and the endpoint.
The first failing request ID and the last succeeding one — the rotation window lives between them.
Your organization ID.
Whether every call fails or traffic is mixed — mixed failure means two sidecar generations disagree and naming that pattern shortens the fix.

Unexpected sidecar error

VOICE_BIOMETRICS_ERROR (503) is the fallthrough: the sidecar answered, but its answer was unusable — an HTTP status outside the mapped set, a 200 with a non-JSON body, or a 200 body that fails validation (a missing or malformed embedding, spoof_score, or model_versions field). The validation gate exists deliberately: a partial answer must never slip past the anti-spoof check as a silent pass, so a malformed body fails loudly as a 503 instead of degrading into a weak verdict.

Cause table

Symptom	Likely cause	Fix
Unexpected HTTP status in the message (`returned 502`, `returned 500`)	The sidecar crashed mid-request or is erroring on inference	Retry once; persistent occurrences are platform-side
`non-JSON body on a 200 response`	A proxy or crash page between the API and the sidecar	Always platform-side — escalate directly
`malformed /analyze body (missing or invalid fields)`	Sidecar model or serialization drift	Always platform-side — escalate directly

Log capture

Keep the full error message — it carries the offending status or the exact field list the validator rejected (embedding, spoof_score, and so on), and that string is the entire diagnostic. Two or three entries with their request IDs and timestamps are enough; there is no tenant-side packet capture or sidecar log you can pull yourself.

Envelope sample

{
  "error": {
    "code": "VOICE_BIOMETRICS_ERROR",
    "message": "Voice biometrics sidecar returned a malformed /analyze body (missing or invalid fields): spoof_score",
    "status": 503
  },
  "meta": {
    "request_id": "req_vbs_err1",
    "timestamp": "2026-09-12T10:02:44Z"
  }
}

Ticket checklist

The code (VOICE_BIOMETRICS_ERROR) and the endpoint.
The full message string, verbatim — the status number or field list inside it is the diagnosis.
Request IDs and timestamps for the failing calls.
Your organization ID.
How long the failures persisted across your retries.

Configuration

CONFIGURATION_ERROR (500) with the DEVOTEL_VOICE_BIOMETRICS_INTERNAL_SECRET wording fires before any sidecar call is attempted: the deployment is missing the internal signing secret, so no request can be signed at all. Every enroll and every verify fails on it, from the moment the configuration drifted, for every tenant on the deployment. This is operator-side — the fix is to restore the secret on the platform configuration, and there is nothing tenant-side to tune, rotate, or re-send. When you see it, treat it as an incident rather than a runbook step: capture one envelope and escalate immediately. It never resolves through your retry policy, and it is the one code on this page where a green dashboard panel alongside red API results is expected — the panel reads stored settings; the calls fail at signing before verification could run.

Fallback semantics — what a failure does to your flow

A 503 or 500 from this page’s codes returns no decision at all — no similarity score, no spoof verdict, nothing to branch a caller journey on. Decide the outage posture at your integration boundary, before it happens:

Fail-closed: hold or route to an agent. Treat a retry-exhausted 503 the way you treat result: "failed_no_match" — the caller does not get the verified path. This fits high-assurance flows (payment release, account recovery) where an unverified speaker must not pass. Queue the caller for a manual check or an agent rather than rejecting outright.
Fail-open into step-up. When the verify outcome is weak or unavailable, fall through to a second factor — the auto_2fa_on_low_confidence flag on Verify profiles and fallback chains is the in-product version of this pattern (it signals step-up on a weak verdict; the same branch point can consume a 503 instead of a weak verdict). An OTP channel is the usual second rung.
Do not fail-open into acceptance. A connectivity failure is not a match — an integration that maps “verify errored” to “caller verified” converts every sidecar outage into a bypass. The one exception is deliberate and logged: a documented operator choice for a specific low-risk flow, never the default.

Whichever posture you pick, log the posture you took. When we pull a failure window apart, “you accepted N callers during the outage” versus “you held N callers” is the first question the timeline has to answer.

What NOT to try

Do not retry VOICE_BIOMETRICS_AUTH_FAILED or CONFIGURATION_ERROR in a loop. Both fail identically on every attempt — the drift is on the signing side, not in your request.
Do not treat a 503 as a match failure. A failed_no_match verdict means the sidecar ran and the speaker scored below your threshold; a 503 means the sidecar never scored anything. Branching a 503 into your “reject” path is correct only if fail-closed is your chosen posture — and even then, log it as an outage, not as a rejection.
Do not switch off the liveness challenge to “get calls through”. If POST /challenge returns VOICE_BIOMETRICS_UNAVAILABLE, sending verify without challenge fields is a supported degradation — the route says so in its own message — but it drops the replay-resistance gate. Restore the challenge flow when the store recovers.
Do not send longer samples to beat a timeout. The 10-second budget is fixed; longer audio only spends more of it on inference. Trim to the spoken phrase.

Reference

Troubleshooting: voice biometrics failures

Troubleshooting: voice biometrics failures

Code matrix — symptom to fix

Sidecar connectivity

Cause table

Retry-safety

Envelope sample

Ticket checklist

Sidecar auth

Secret rotation checklist

Envelope sample

Ticket checklist

Unexpected sidecar error

Cause table

Log capture

Envelope sample

Ticket checklist

Configuration

Fallback semantics — what a failure does to your flow

What NOT to try

See also

​Troubleshooting: voice biometrics failures

​Code matrix — symptom to fix

​Sidecar connectivity

​Cause table

​Retry-safety

​Envelope sample

​Ticket checklist

​Sidecar auth

​Secret rotation checklist

​Envelope sample

​Ticket checklist

​Unexpected sidecar error

​Cause table

​Log capture

​Envelope sample

​Ticket checklist

​Configuration

​Fallback semantics — what a failure does to your flow

​What NOT to try

​See also

Troubleshooting: voice biometrics failures

Code matrix — symptom to fix

Sidecar connectivity

Cause table

Retry-safety

Envelope sample

Ticket checklist

Sidecar auth

Secret rotation checklist

Envelope sample

Ticket checklist

Unexpected sidecar error

Cause table

Log capture

Envelope sample

Ticket checklist

Configuration

Fallback semantics — what a failure does to your flow

What NOT to try

See also