Skip to main content

​
BYO LLM inference credential

Most organizations use the curated provider tiers from the LLM provider preference guide — the free Devotel base tier or the premium Anthropic tier — and never manage a provider key. Some organizations have to own the credential that agent inference runs against: their security team holds a direct Anthropic contract, or their data-residency policy requires inference to stay inside a specific region. For those organizations, Orbit’s BYO LLM inference credential surface puts agent inference under a credential you own. The BYO LLM Inference Credential page lives at Settings → Agents → LLM provider credential and is guarded to owner and admin roles. Until you register one, the organization runs on the curated provider lineup from the LLM provider preference guide — the two guides cover the two positions, and this page picks up where that one ends.

​
Privacy: the framing that decides whether you need BYO at all

The prompt and conversation content your agents generate is briefed to one inference endpoint per organization at a time. There is no per-agent provider fan-out: whatever is authoritative for your organization — the platform credential, your recorded preference, or your BYO credential — receives the full briefing, and nothing else does. So the privacy question is never “how many vendors see this”; it is “whose account receives it, and in which region.” This guide pairs the WHAT with that WHY. Each property named below has a privacy or governance consequence — an account boundary, an egress location, a retention posture, or a fallback exit. When a field is collected for provenance rather than for routing (the key alias, the justification), the guide says so. If your security review only needs the curated tiers’ posture, the preference guide is the whole story; if it requires the account boundary drawn under your own contract or region, this page is the control plane.
Every step below is a tenant-owned control: you pick the provider, region, and endpoint; you register, activate, rotate, and revoke the posture. No step is gated or mandated by the platform — and until you activate and enforce a credential, the platform-owned key stays authoritative, exactly as it does today.

​
1. What a BYO inference credential is

A BYO inference credential records where your organization’s agent inference should run and whose account should bill for it. The credential catalog supports exactly three Anthropic-native transports, all serving the same Claude models: Because all three serve the same Claude models, registering a credential never introduces a second model vendor — your agents behave identically; the difference is whose account the inference egresses from and which region it egresses in. Four properties define the model, and each carries a privacy or governance consequence:
  • The secret is stored encrypted at rest and never echoed back. Orbit fingerprints the credential on submission and stores it under the platform encryption envelope; read responses only ever return the fingerprint. The consequence for your security review: Orbit never exposes your provider secret over its API, and a leaked dashboard session cannot disclose it. You verify which credential is in use by matching fingerprints out-of-band, never by reading the secret back.
  • The config surface makes no provider network call. Register is a control-plane recording: Orbit validates the endpoint’s shape (a public https host when a base URL is required) and fingerprints it. The implication for a privacy review is that no prompt or conversation content leaves the platform until you activate and enforce — the live routing layer reads the config and routes through your endpoint only once it is active + enforced. Registering early to satisfy a procurement deadline changes nothing on the data-path.
  • Platform-owned key stays authoritative until enforced. Inference continues to resolve on the platform credential (or your recorded preference) until you activate with enforcement — so a pending credential can be reviewed, fingerprint-checked, and approved by a second officer with zero exposure change.
  • Residency and governance stay a per-organization choice. The credential record lives in your organization settings and never crosses the account boundary into another organization, so a concurrent write to another settings key never clobbers it and no neighboring tenant shares any part of your credential’s routing.
A credential moves through one state machine, and every transition writes an audit-log entry:

​
2. What crosses the wire under each posture

The whole privacy evaluation reduces to one question: when my organization is in this state, whose endpoint receives my agents’ briefing, and in which region? Map it once and the review writes itself: Three boundaries the table makes concrete:
  • Account boundary. On the curated tiers, Orbit is the account of record with the provider. On an enforced BYO credential, your organization is — the workspace content is directly briefed to your account’s endpoint, and Orbit does not process it for the transmission beyond routing it.
  • Regional boundary. Egress location follows your registered region and gateway. If your policy names a region, register the endpoint in that region — the control plane records the declared region for provenance and your gateway is what physically keeps inference there.
  • Retention boundary. Orbit retains configuration and audit metadata about the credential (provider, fingerprint, timestamps, justification); it does not retain your raw secret in plaintext, and it has no persistence role for the inference payloads your endpoint receives — retention of those is governed by your own provider agreement.

​
3. When BYO fits

Pick BYO when either of these holds; otherwise the curated preference tiers are the simpler surface:
  • Direct Anthropic contract. Your security or finance team requires inference billed under your own Anthropic agreement rather than the platform’s partner account.
  • Regional residency. Your data-governance policy requires inference to stay inside a specific region — for example a Bedrock gateway in eu-central-1, or a Vertex gateway in europe-west1. The registered region is recorded on the credential for provenance.
BYO does not buy you a different model set (the three transports serve the same Claude models), and it does not change agent behavior — it changes whose account receives the bill and where the inference egresses.

​
4. Register a credential

Open Settings → Agents → LLM provider credential. You need owner or admin role.
  1. Provider — Anthropic, AWS Bedrock, or Google Vertex. The form hints update to the provider you pick.
  2. Region — the region the inference should egress in (e.g. eu-central-1). Why we collect it: recorded on the credential for residency provenance, so your audit trail shows the declared egress region next to the fingerprint.
  3. Endpoint URL — required for Bedrock and Vertex (the in-region gateway URL), optional for a direct Anthropic key (leave blank to use the global Anthropic endpoint). Why we collect it: the routing target once enforcement is on — this is the address your content will be briefed to, so the control plane validates it is a public https host before recording it.
  4. Credential transport — api_key (sent as x-api-key) or auth_token (sent as a bearer token on Authorization). Why we collect it: the transport changes how your secret is presented to your endpoint — pick the one your gateway expects.
  5. Credential secret — paste the API key or bearer token (minimum 8 characters). Stored encrypted at rest; only its fingerprint is ever shown after saving. Why we collect it: the only field that is sensitive at rest — encrypted immediately, never shown again, never included in any read response.
  6. Key alias (optional) — a human-readable label shown wherever the fingerprint would be cryptic. Why we collect it: provenance for humans — the alias lets an operator answer “which of our keys is this?” without ever handling the secret.
  7. Justification (optional) — free text recorded with the registration; cite the policy or clause that requires BYO (e.g. a data-governance addendum). Why we collect it: governance record, never used for routing — it is what your auditor reads next to the fingerprint when reconstructing who authorized this posture and why.
The curl equivalent:
Submit Register credential. The credential lands in pending state and one audit-log entry is written. Registering while a credential is live is refused with 409 LLM_CREDENTIAL_ALREADY_REGISTERED — use rotate for a live credential, or revoke first.
Register is only available when no credential exists or the previous one was revoked — while a credential is pending or active, the register card is hidden and the server refuses with 409 LLM_CREDENTIAL_ALREADY_REGISTERED.
The GET read surface returns the current config, the provider catalog, and the resolved posture (whether inference should route through the credential):

​
5. Activate, and what enforcement flips

From pending, Activate credential — with the optional Enforce immediately checkbox. While the credential is pending, the page shows its fingerprint so your operator can confirm it matches the credential you shared out-of-band before you flip it live.
  • Activate without enforcement keeps the registration advisory: the platform-owned key remains authoritative and inference continues as before.
  • Activate with Enforce immediately makes the credential the authoritative route for this organization’s agent inference — subsequent agent turns route through your own endpoint. Enforcement holds until you revoke the credential.
Only a pending credential can be activated — activating an active or revoked one returns 409 LLM_CREDENTIAL_NOT_PENDING. Every activation writes an audit entry with the fingerprint and the enforcement flag.

​
6. Rotate with no downtime

From pending or active, select Rotate credential and submit the replacement payload in the same form as registration:
Rotation preserves the lifecycle state and no downtime:
  • An active credential stays active and its enforcement is preserved — no re-registration, no downtime window.
  • The rotation count increments for provenance, so your audit trail shows each replacement as a first-class event.
  • Rotating a revoked credential is refused with 409 LLM_CREDENTIAL_REVOKED — register a fresh one instead.

​
7. Revoke deliberately

Revoke is the deliberate off-boarding step — a compromised key, a contract termination, or moving back to the platform key:
  1. Select Revoke credential, optionally record a justification, and confirm in the destructive-action dialog.
  2. Revoking clears enforcement immediately — agent inference falls back to the platform-owned key for new turns.
Re-enabling BYO inference after a revocation requires registering a credential afresh (Section 3); revoking an already-revoked credential returns 409 LLM_CREDENTIAL_ALREADY_REVOKED.

​
8. Fallback policy and interaction with the provider preference

Two knobs decide what happens when things go wrong, and each has a privacy consequence:
  • Pooled fallback (off by default). Toggle Fall back to Orbit’s pooled model on repeated failure on the config card, or POST /api/v1/agents/llm-provider-credential/pooled-fallback with { "enabled": true }. When enabled, if your endpoint fails repeatedly, agent turns route through Orbit’s pooled model until it recovers, instead of failing outright. Why it defaults to off: enabling it means your content can egress under Orbit’s account during a failure window — the opposite of the account boundary you registered the credential for. A residency or cost-governance credential never leaves your endpoint unless you explicitly accept that trade-off.
  • Preference interplay. An active + enforced BYO credential overrides the curated LLM provider preference for this organization — the preference record is untouched (it stays a tenant-level setting) and resolves again the moment the credential is revoked. This is the deliberate exit ramp: revoke, and the preference (or platform default) takes back over with no additional step — the boundary moves back without a leftover record pointing at your endpoint.

​
9. Audit and roles

Every register, activate, rotate, revoke, and pooled-fallback toggle is gated to owner and admin and writes an audit-log entry with the provider, fingerprint, enforcement flag, rotation count, and (for revokes) your justification. The page itself is wrapped in a role guard so developers and members cannot even load the config view. Filter Settings → Audit log by the credential fingerprint to reconstruct the full lifecycle — see the audit log guide for filtering and export. Track the spend split after activation. Once the credential is enforced, the /insights/llm-spend dashboard is where the routing change becomes visible: tokens and cost that flowed through the platform credential drain on its provider line while your own endpoint accrues. Read the walkthrough in Read the LLM spend dashboard for the per-agent, per-model, and per-conversation breakdowns, and the spend alarms that keep a runaway agent from landing on your month-end statement.

​
10. Failure modes

  • Malformed endpoint — a non-https or unreachable-looking base URL is refused at register or rotate time with a 422 validation error that names the field.
  • Conflict codes — register over a live credential (409 LLM_CREDENTIAL_ALREADY_REGISTERED), activate a non-pending credential (409 LLM_CREDENTIAL_NOT_PENDING), rotate or toggle fallback on a revoked credential (409 LLM_CREDENTIAL_REVOKED), or revoke twice (409 LLM_CREDENTIAL_ALREADY_REVOKED) — each returns a named conflict so an automation can branch on it.
  • Role gating — a non owner/admin caller gets 403 on writes; the read inherits the agents read scope of the parent auth encapsulation.

​
See also

  • LLM provider preference — the curated free/premium tier the BYO credential overrides, and which takes back over on revoke
  • Read the LLM spend dashboard — where the per-provider spend split shows after activation
  • Audit log — filter the fingerprint’s full lifecycle
  • Customer-Managed Keys (BYOK) — the sibling key-management control plane. Different scope: it governs KMS keys that encrypt your data at rest, not LLM inference — this guide’s credentials choose whose account runs agent inference and which region it egresses in. For the KMS scan scope, see BYOK customer-managed keys
  • Organization API — endpoint shapes for the preference endpoints the BYO credential interacts with