Troubleshooting: prompt-regression rejects an agent-version rollback
When you promote an older agent version back to live (the rollback call), a prompt-regression evaluator may gate it. The evaluator rejects target versions that fall outside its registered bounds — for example a version whose prompt diff it has never scored, or one whose eval-resolved verdict is not a plain pass/fail. A rejected rollback register as theprompt-regression evaluator on the version; without the resolution
below, the rollback never applies.
If the prompt-regression evaluator registered on your version, follow the
steps below — you only need the agent’s version ids and an API key.
1. Confirm the symptom
A blocked rollback shows one tell: the evaluator registered asprompt-regression on the version row, and the promote call either returns a
422 or queues without ever moving the live pointer. That is the evaluator
saying “I cannot write a verdict for this target,” not a general rollout
failure — the canary/status surfaces will look empty because the version
never became the rollback target the current live agent assumed.
2. Locate the evaluator that attached
Search your agent-version tooling for theprompt-regression evaluator id
and read its registered bounds. A prompt-regression evaluator works like a
lint rule: it approves a rollback target only when the version is inside the
set it was registered against (a scaffolded harness, for instance, cannot
verdict a plain codebase access). Confirm the target version id is exactly
the one you intend — a mis-typed or drifted id is the most common cause.
Read the version row and the eval-run history over the API to confirm which
verdict registered — and against which version id:
GET /api/v1/agents/{agentId}/versions/{versionId}returns the exact version row you are rolling back to — compare theidandversion_numberyou send into the rollback with the row you meant.GET /api/v1/agents/{agentId}/evals/runslists recent runs (filter with?status=completed); the newest run whose dataset covered the rollback target is the verdict prompt-regression registered. If no run covers that target version id, that gap is the registration.GET /api/v1/agents/{agentId}/evals/runs/{runId}reads one run’s verdict (passed/failedplus the aggregate pass rate) — a run whose status never reached a plain pass/fail is the class this page resolves.
3. Re-run the canary evaluation as the rollback run
Send the rollback to current semantics by issuing the eval the pipeline expects: re-run the canary evaluation against the target version. A fresh run replaces the stale registered verdict with a real verdict and clears the register. Re-running also covers the second failure mode: an evaluator that returned a non-plain output (anything but a clean pass/fail) is replaced by a fresh verdict, which the rollback gate can consume. Trigger the re-run with the same call the datasets → runs loop uses — a new run against the dataset that covers your rollback target:202 with status: "pending" — the run executes in the
background. Poll GET /api/v1/agents/{agentId}/evals/runs/{runId} until
status flips to completed, then read the aggregate verdict
(passed_rows, failed_rows, avg_score). That completed verdict is what
the rollback gate needs from step 2’s register — if the run comes back
failed, fix the version first instead of approving a dirty rollback.
4. Approve the cleared rollback
Once the re-run returns a plain verdict, approve the rollback explicitly. Until an approval lands, the rollback stays pending no matter how clean the verdict is. Reject instead when the rollback target is genuinely wrong — approval is withheld deliberately on a dirty verdict, so do not approve a target that failed its re-run. The approval is the promote call itself — promote the rollback target as you would any version, now that the gate has a clean verdict to consume:422 on this call means a gate
still saw a regression — read the gate report in the response body, which
names the failing run, before retrying. To decline the rollback instead,
leave it pending: the live pointer stays on the current version.
5. How it composes with the datasets loop
The rollback path is one stage of the datasets → runs → triage → gate loop: prompt-regression registers during that loop, and re-running the canary eval is re-entering the loop at its gate stage. Keep your rollback candidates coming from saved versions inside that loop — every version the loop evaluated is a version the rollback gate can clear, and versions captured outside the loop are the ones that registerprompt-regression.
See also
- Agent versions: snapshot, stage, and roll back — the version lifecycle the rollback moves through.
- Agent evals: datasets, runs, and pass-rate gates — the loop that registers prompt-regression.
- Safely roll out an AI agent — the release pipeline rollback protection gates.
- Troubleshooting: agent eval-queue failures — when the eval run itself is what is stuck.