Skip to main content

​
Supervise pending tool approvals

A tool your agent calls is either unguarded (it runs as soon as the model seeks it) or approval-gated (it pauses until a human signs off). This guide walks the full supervisor loop for gated tools: declare which tools require approval, watch the pending queue, decide approve or deny, and read the audit trail. The decision recurs hourly in production, so the loop is what matters — not the one-time setup. The endpoint paths below are relative. Send them against https://api.orbit.devotel.io/api/v1. The concepts behind the queue are in Human-in-the-loop oversight; this guide is the operational walk.

​
1. When gate tracking acts: pre-execution vs post-hoc

The approval gate is synchronous pre-execution: when the model emits a gated tool call, the executor writes a pending approval row and returns a blocked stub to the model instead of running the tool. Nothing touches the action until a human approves it. After approval the resume worker runs the deferred call and the conversation continues. That is the only safe timing that meets “a human saw this first”. Async post-hoc review is the opposite shape and is cheaper: let the tool run now and review the record after. Text-only guardrail violations (warn-severity) work like that, because a wrong reply is recoverable. For a gated tool call, post-hoc is wrong — a payment capture or a PII egress has already happened by the time you review it. Choose pre-execution approval for the calls that hurt; choose post-hoc warning for the rest. The worked pattern in section 8 shows both in one agent.

​
2. Prerequisites

  • A supervisor identity. Approve and reject calls run behind the owner or admin role — reading the list or the dashboard view is fine for a supervisor role, but the decide step requires owner/admin. Authenticate every call with an API key (X-API-Key: dv_live_sk_...) or a dashboard session.
  • Tool policy in Agent Studio. The agent’s tools are declared in Agent Studio with a per-tool confirmation flag, or through the agent API’s safety_config. Either path marks a tool as gated; both feed the same queue.
  • Your tools exist. The gate fires on a tool the agent actually has. A typo in the tool id silently produces an unguarded call — verify the id before you rely on the gate.

​
3. Declare which tools require approval

Gate a tool per-call by setting its confirmation to "always" in the tool definition. Gate every tool on the agent by setting safety_config.approval_required: true. The two flags are additive; either one firing puts the call in the queue.
Prefer per-tool gating. approval_required: true stops every tool call and quickly buries the queue under harmless lookups — see the worked pattern in section 8.

​
4. Surface pending approvals to a supervisor

The pending queue lives on the dashboard at /agents → the tool-approvals tab, and programmatically at GET /api/v1/agents/tool-approvals?status=pending&limit=50:
Each row carries the tool the agent wanted (tool_id), the arguments the model proposed (tool_args), the agent and conversation refs, requested_at, and a cost_estimate when the tool priced the call in advance — an SMS fan-out or a payment capture shows the projected spend right on the pause card. Filter by status (pending, approved, rejected) and age the queue by requested_at. Wire the same GET into your own alerting path (an oncall webhook or a polling supervisor screen) — the JSON shape above is the contract. The badge-count on the tool-approvals tab in the dashboard reads the same pending list, so the queue is visible without manual polling. A row that stays pending is the failure mode to watch: the turn is blocked and the customer experience hangs while no one decides. Treat an aged pending queue as an alarm, not an inbox.

​
5. Resolve with approve or deny

Approve lets the deferred call run. reason is optional on approve:
resumed: true confirms the deferred-turn resume job was accepted. If the resume queue is briefly unavailable the decision still saves (resumed: false + a warning string) and the platform retries — the approval record is the source of truth, no decision is lost. Reject blocks the call. reason is required so the refusal is documented:
Common responses a supervisor sees:

​
6. Timeout policy: what an unresolved pending row means

There is no auto-approve timeout by design: a pending row never resolves itself, and an aged pending row simply pauses the conversation until someone decides. That is the deliberate, safe default — an auto-approve after N minutes would defeat the point of the gate. So the timeout you actually operate is on the queue, not the row: a pending approval that sits past a few minutes is a stalled turn, and the correct action is to escalate the queue to a supervisor, not to let it auto-resolve. The warning string on resumed: false is your signal to route a hard-deny or a manual callback. If your risk posture truly wants an auto-approve-after-N-minutes rule, the fall-back lives outside the gate: add a scheduled sweep that reads the pending list (step 4), rejects or approves with a named reason (step 5), and is explicit in the audit trail (step 7). Run that sweep from your own operator tooling — never inside the agent.

​
7. Audit — every decision writes the supervisor identity

Both decide endpoints append an audit record with the deciding user, the tool id, and whether a reason was attached. The audit entry is written after the status update persists, so the trail never claims a decision that didn’t commit. Approvals carry approved with an optional-reason flag; rejections carry rejected with the reason length recorded (never the free text verbatim, which shouldn’t spill to a warn-path log). Read the trail in the audit log.

​
8. Common pattern — gate high-risk, leave low-risk unguarded

The single most useful shape: gate the few calls that move money or leak data; let everything else run.
  • Gate: payment_capture, issue_refund, pii_egress (export / send), cancel_subscription — anything irreversible or paid.
  • Unguarded: lookup_order_status, search_knowledge_base, get_weather — read-only or recoverable.
Set the gated ones with per-tool confirmation: "always" (step 3). If you set approval_required: true on the agent instead, every safe lookup goes to the queue too and a supervisor spends the shift on trivia. Pick the gate on the high-risk tools, not the agent as a whole. Where a cost_estimate exists on the gated tool, the pause card shows the projected spend — use it to reject the obviously-wrong fan-out before the carrier touches it.