Skip to main content

​
Agent versions

Every endpoint in the rollout surface — canary, regression replay, persona simulation, shadow comparison — operates on a saved version, not on whatever the agent happens to hold right now. This page covers what a version is, how one moves from candidate to production, and how each version-aware endpoint addresses it. It is the prerequisite for the rollout pipeline. Send the endpoints below against https://api.orbit.devotel.io/api/v1.

​
What a saved version is

A version is an immutable snapshot of the agent’s full configuration, captured at a point in time:
  • system_prompt
  • model, temperature, max_tokens
  • tools and knowledge_base_ids
  • safety_config and config
  • max_cost_per_conversation_cents
Each version carries a monotonically increasing version_number, an optional label you set, an optional branch for parallel experiment lines, and a parent_version_id recording what it forked from. Because the snapshot is immutable, a version id is a stable pointer you can hand to a simulation, a canary, or a rollback — the thing you tested is exactly the thing you promoted.

​
The version lifecycle

A version moves through four stages, and you can reverse the last one:
  1. Save. Snapshot either the agent’s current live state or an edited draft. Nothing changes on a save — you have only recorded a candidate.
  2. Candidate. Test the snapshot offline: replay regression tests against it, run persona simulations, or diff it against production.
  3. Stage. Roll the candidate out gradually with a canary rollout, or compare it through a shadow agent.
  4. Promote. Make the version live. Promotion is forward-only: promoting an older snapshot mints a new version that copies it, so the history stays a clean linear timeline and a rollback is itself an auditable entry.

​
Working with versions

Five endpoints manage the snapshots themselves.
  • GET /agents/:id/versions — list saved versions, newest first, with the distinct branch names. Cursor-paginated; pass the returned next_cursor (a version_number) back as cursor.
  • POST /agents/:id/versions — mint a version from the agent’s current state. Body fields are optional: label, branch. This routes through the same role gate as editing the agent (owner, admin, or developer). If the current prompt differs from the last snapshot it becomes a new checkpoint; if nothing changed it returns the existing row.
  • POST /agents/:id/versions/:vid/branch — fork a version into a named experiment line without touching the live agent. Promote the branch when it wins.
  • POST /agents/:id/versions/:vid/promote — set that version live. The snapshot’s fields are copied onto the agent, the agent’s version counter advances, and the promotion lands in your change log. This is also the rollback call: promote an older version and you have rolled back, with a fresh version row marking it.
  • GET /agents/:id/versions/:vid/diff — compare one version against another (pass baseline_version_id) or against the agent’s current live config when the baseline is omitted.
Two aliases exist for operators who think in prompt terms: GET /agents/:id/prompt-history (same list as versions) and POST /agents/:id/prompt-rollback/:vid (same action as promote, audit-logged as a rollback). Version ids are scoped to their agent. Every endpoint below validates that a version_id belongs to the agent in the path and returns a 404/422 otherwise — a candidate can never be borrowed from a neighbor’s agent, even inside your own workspace.

​
Regression tests on versions

Regression tests turn “the agent behaved on Tuesday” into a replayable check you run on every change.
  • POST /agents/:id/regression-tests — save a scripted conversation (conversation_json), the expectations for the reply (expected_outputs), and optionally a pinned_version_id the test replays against by default.
  • GET /agents/:id/regression-tests — list saved tests (up to the newest 100).
  • DELETE /agents/:id/regression-tests/:testId — drop one.
  • POST /agents/:id/regression-tests/run-all — replay every saved test and report pass, fail, or error per test with a total summary.
A run resolves its target prompt in this order: a body field agent_version_id on run-all overrides every pin for that one run; otherwise each test falls to its own pinned_version_id; a stale pin falls through to the live config. Replays run in sandbox mode — no billing, no memory writes — and never mutate the live agent, so you can replay the whole corpus against a candidate before it serves a single customer. Save your hardest cases: the refund that must cite policy, the turn that must stay in scope, the escalation that must hand off. A test that passes here is a gate the rollout can trust.

​
Persona simulation against a candidate

POST /agents/:id/persona-simulation grades a candidate before it meets traffic. Each scenario spawns a scripted customer persona that reacts turn by turn to the agent’s real replies; a judge then scores the conversation against the scenario’s objective and rubric. Pass candidate_version_id to point the run at a saved version — omit it and the live config is simulated. The endpoint is rate-limited tightly (each scenario fans out to persona, runtime, and judge calls), so send a short curated set rather than a full corpus. A scenario that fails here never reaches a canary.

​
The canary ladder

POST /agents/:id/canary-rollout starts a staged traffic shift onto a candidate:
  • candidate_version_id (required) — the snapshot taking traffic. baseline_version_id (optional) pins what the candidate is compared against; omit it and the current live version is the baseline.
  • stages — the percent of traffic per stage. The platform default is 5 → 25 → 50 → 100; pass your own array to move faster, slower, or in finer steps (up to 20 stages, each 1–100%).
  • min_stage_sample_size — how many conversations a stage must collect before its scorecard can advance it.
Four operations close the loop:
  • GET /agents/:id/canary-rollout — current stage and its live scorecard verdict.
  • POST /agents/:id/canary-rollout/evaluate — score the stage against its gates and apply the decision: advance, hold, rollback, or complete.
  • POST /agents/:id/canary-rollout/rollback — pull the candidate regardless of the gates, pinning the baseline back.
  • POST /agents/:id/canary-rollout/cancel — void a rollout started by mistake.
Rollout state is persisted, so an evaluator that runs between your calls still sees the same stage and candidate — nothing needs to be resupplied.

​
Relation to the release pipeline

Versions are one stage in a longer pipeline: pick models, load knowledge sources, define guardrails, then test offline, stage on a canary, and monitor the scorecard. The rollout pipeline guide wires each of these endpoints into an ordered release with a monitored rollback path — read it once you are comfortable with the version lifecycle here.

​
Limits and rules

  • Version-aware only. Canary, regression replay, persona simulation, and shadow promotion all resolve a saved version id. There is no flag that canaries “whatever the agent currently holds” — save the version first, then point the endpoint at it.
  • Immutable snapshots. A version never mutates after it is saved; promote and rollback move the live pointer, the history rows stay fixed.
  • Per-agent scope. A version id is valid only for the agent that produced it.
  • Scale caps. Version lists paginate; regression lists return the newest 100 tests and run-all replays the oldest 25 per run (5 runs/minute); persona simulation accepts a small curated scenario set per call.
  • Roles. Listing and diffing need read access; saving versions, branching, and regression writes need owner, admin, or developer; promote and rollback need owner or admin.
  • Tenant-only controls. Everything on this page is scoped to your workspace and your agents — no version, rollout, or regression artifact crosses tenant boundaries.

​
See also