Skip to main content

​
BYO LLM inference credential

Most organizations use the curated provider tiers from the LLM provider preference guide — the free Devotel base tier or the premium Anthropic tier — and never manage a provider key. Some organizations have to own the credential that agent inference runs against: their security team holds a direct Anthropic contract, or their data-residency policy requires inference to stay inside a specific region. For those organizations, Orbit’s BYO LLM inference credential surface puts agent inference under a credential you own. The BYO LLM Inference Credential page lives at Settings → Agents → LLM provider credential and is guarded to owner and admin roles.
Every step below is a tenant-owned control: you pick the provider, region, and endpoint; you register, activate, rotate, and revoke the posture. No step is gated or mandated by the platform — and until you activate and enforce a credential, the platform-owned key stays authoritative, exactly as it does today.

​
1. What a BYO inference credential is

A BYO inference credential records where your organization’s agent inference should run and whose account should bill for it. The credential catalog supports exactly three Anthropic-native transports, all serving the same Claude models: Because all three serve the same Claude models, registering a credential never introduces a second model vendor — your agents behave identically; the difference is whose account the inference egresses from and which region it egresses in. Four properties define the model:
  • The secret is stored encrypted at rest and never echoed back. Orbit fingerprints the credential on submission and stores it under the platform encryption envelope; read responses only ever return the fingerprint, so you can confirm which credential is in use without exposing it.
  • The config surface makes no provider network call. Register is a control-plane recording: Orbit validates the endpoint’s shape (a public https host when a base URL is required) and fingerprints it. The live routing layer reads the config and routes through your endpoint only once it is active + enforced.
  • Platform-owned key stays authoritative until enforced. Inference continues to resolve on the platform credential (or your recorded preference) until you activate with enforcement.
  • Residency and governance stay a per-organization choice. The credential record lives in your organization settings, so a concurrent write to another settings key never clobbers it.
A credential moves through one state machine, and every transition writes an audit-log entry:

​
2. When BYO fits

Pick BYO when either of these holds; otherwise the curated preference tiers are the simpler surface:
  • Direct Anthropic contract. Your security or finance team requires inference billed under your own Anthropic agreement rather than the platform’s partner account.
  • Regional residency. Your data-governance policy requires inference to stay inside a specific region — for example a Bedrock gateway in eu-central-1, or a Vertex gateway in europe-west1. The registered region is recorded on the credential for provenance.
BYO does not buy you a different model set (the three transports serve the same Claude models), and it does not change agent behavior — it changes whose account receives the bill and where the inference egresses.

​
3. Register a credential

Open Settings → Agents → LLM provider credential. You need owner or admin role.
  1. Provider — Anthropic, AWS Bedrock, or Google Vertex. The form hints update to the provider you pick.
  2. Region — the region the inference should egress in (e.g. eu-central-1).
  3. Endpoint URL — required for Bedrock and Vertex (the in-region gateway URL), optional for a direct Anthropic key (leave blank to use the global Anthropic endpoint).
  4. Credential transport — api_key (sent as x-api-key) or auth_token (sent as a bearer token on Authorization).
  5. Credential secret — paste the API key or bearer token (minimum 8 characters). Stored encrypted at rest; only its fingerprint is ever shown after saving.
  6. Key alias (optional) — a human-readable label shown wherever the fingerprint would be cryptic.
  7. Justification (optional) — free text recorded with the registration; cite the policy or clause that requires BYO (e.g. a data-governance addendum).
The curl equivalent:
Submit Register credential. The credential lands in pending state and one audit-log entry is written. Registering while a credential is live is refused with 409 LLM_CREDENTIAL_ALREADY_REGISTERED — use rotate for a live credential, or revoke first.
Register is only available when no credential exists or the previous one was revoked — while a credential is pending or active, the register card is hidden and the server refuses with 409 LLM_CREDENTIAL_ALREADY_REGISTERED.
The GET read surface returns the current config, the provider catalog, and the resolved posture (whether inference should route through the credential):

​
4. Activate, and what enforcement flips

From pending, Activate credential — with the optional Enforce immediately checkbox. While the credential is pending, the page shows its fingerprint so your operator can confirm it matches the credential you shared out-of-band before you flip it live.
  • Activate without enforcement keeps the registration advisory: the platform-owned key remains authoritative and inference continues as before.
  • Activate with Enforce immediately makes the credential the authoritative route for this organization’s agent inference — subsequent agent turns route through your own endpoint. Enforcement holds until you revoke the credential.
Only a pending credential can be activated — activating an active or revoked one returns 409 LLM_CREDENTIAL_NOT_PENDING. Every activation writes an audit entry with the fingerprint and the enforcement flag.

​
5. Rotate with no downtime

From pending or active, select Rotate credential and submit the replacement payload in the same form as registration:
Rotation preserves the lifecycle state and no downtime:
  • An active credential stays active and its enforcement is preserved — no re-registration, no downtime window.
  • The rotation count increments for provenance, so your audit trail shows each replacement as a first-class event.
  • Rotating a revoked credential is refused with 409 LLM_CREDENTIAL_REVOKED — register a fresh one instead.

​
6. Revoke deliberately

Revoke is the deliberate off-boarding step — a compromised key, a contract termination, or moving back to the platform key:
  1. Select Revoke credential, optionally record a justification, and confirm in the destructive-action dialog.
  2. Revoking clears enforcement immediately — agent inference falls back to the platform-owned key for new turns.
Re-enabling BYO inference after a revocation requires registering a credential afresh (Section 3); revoking an already-revoked credential returns 409 LLM_CREDENTIAL_ALREADY_REVOKED.

​
7. Fallback policy and interaction with the provider preference

Two knobs decide what happens when things go wrong:
  • Pooled fallback (off by default). Toggle Fall back to Orbit’s pooled model on repeated failure on the config card, or POST /api/v1/agents/llm-provider-credential/pooled-fallback with { "enabled": true }. When enabled, if your endpoint fails repeatedly, agent turns route through Orbit’s pooled model until it recovers, instead of failing outright. A residency or cost-governance credential never leaves your endpoint unless you explicitly accept the trade-off.
  • Preference interplay. An active + enforced BYO credential overrides the curated LLM provider preference for this organization — the preference record is untouched (it stays a tenant-level setting) and resolves again the moment the credential is revoked. This is the deliberate exit ramp: revoke, and the preference (or platform default) takes back over with no additional step.

​
8. Audit and roles

Every register, activate, rotate, revoke, and pooled-fallback toggle is gated to owner and admin and writes an audit-log entry with the provider, fingerprint, enforcement flag, rotation count, and (for revokes) your justification. The page itself is wrapped in a role guard so developers and members cannot even load the config view. Filter Settings → Audit log by the credential fingerprint to reconstruct the full lifecycle — see the audit log guide for filtering and export. Track the spend split after activation. Once the credential is enforced, the /insights/llm-spend dashboard is where the routing change becomes visible: tokens and cost that flowed through the platform credential drain on its provider line while your own endpoint accrues. Read the walkthrough in Read the LLM spend dashboard for the per-agent, per-model, and per-conversation breakdowns, and the spend alarms that keep a runaway agent from landing on your month-end statement.

​
9. Failure modes

  • Malformed endpoint — a non-https or unreachable-looking base URL is refused at register or rotate time with a 422 validation error that names the field.
  • Conflict codes — register over a live credential (409 LLM_CREDENTIAL_ALREADY_REGISTERED), activate a non-pending credential (409 LLM_CREDENTIAL_NOT_PENDING), rotate or toggle fallback on a revoked credential (409 LLM_CREDENTIAL_REVOKED), or revoke twice (409 LLM_CREDENTIAL_ALREADY_REVOKED) — each returns a named conflict so an automation can branch on it.
  • Role gating — a non owner/admin caller gets 403 on writes; the read inherits the agents read scope of the parent auth encapsulation.

​
See also