Troubleshooting: prompt-regression rejects an agent-version rollback
When you promote an older agent version back to live (the rollback call), a prompt-regression evaluator may gate it. The evaluator rejects target versions that fall outside its registered bounds — for example a version whose prompt diff it has never scored, or one whose eval-resolved verdict is not a plain pass/fail. A rejected rollback register as theprompt-regression evaluator on the version; without the resolution
below, the rollback never applies.
If the prompt-regression evaluator registered on your version, follow the
steps below — you only need the agent’s version ids and an API key.
1. Confirm the symptom
A blocked rollback shows one tell: the evaluator registered asprompt-regression on the version row, and the promote call either returns a
422 or queues without ever moving the live pointer. That is the evaluator
saying “I cannot write a verdict for this target,” not a general rollout
failure — the canary/status surfaces will look empty because the version
never became the rollback target the current live agent assumed.
2. Locate the evaluator that attached
Search your agent-version tooling for theprompt-regression evaluator id
and read its registered bounds. A prompt-regression evaluator works like a
lint rule: it approves a rollback target only when the version is inside the
set it was registered against (a scaffolded harness, for instance, cannot
verdict a plain codebase access). Confirm the target version id is exactly
the one you intend — a mis-typed or drifted id is the most common cause.
3. Re-run the canary evaluation as the rollback run
Send the rollback to current semantics by issuing the eval the pipeline expects: re-run the canary evaluation against the target version. A fresh run replaces the stale registered verdict with a real verdict and clears the register. Re-running also covers the second failure mode: an evaluator that returned a non-plain output (anything but a clean pass/fail) is replaced by a fresh verdict, which the rollback gate can consume.4. Approve the cleared rollback
Once the re-run returns a plain verdict, approve the rollback explicitly. Until an approval lands, the rollback stays pending no matter how clean the verdict is. Reject instead when the rollback target is genuinely wrong — approval is withheld deliberately on a dirty verdict, so do not approve a target that failed its re-run.5. How it composes with the datasets loop
The rollback path is one stage of the datasets → runs → triage → gate loop: prompt-regression registers during that loop, and re-running the canary eval is re-entering the loop at its gate stage. Keep your rollback candidates coming from saved versions inside that loop — every version the loop evaluated is a version the rollback gate can clear, and versions captured outside the loop are the ones that registerprompt-regression.
See also
- Agent versions: snapshot, stage, and roll back — the version lifecycle the rollback moves through.
- Agent evals: datasets, runs, and pass-rate gates — the loop that registers prompt-regression.
- Safely roll out an AI agent — the release pipeline rollback protection gates.
- Troubleshooting: agent eval-queue failures — when the eval run itself is what is stuck.