> ## Documentation Index
> Fetch the complete documentation index at: https://docs.orbit.devotel.io/llms.txt
> Use this file to discover all available pages before exploring further.

# Computer-Use Browser Tools

> Give an agent a tool that drives a web UI through a bounded screenshot-step loop, validate guardrails with a dry-run, and reconstruct every run from the audit trail.

# Computer-Use Browser Tools

A **computer-use tool** lets an agent drive a real web UI — navigate, click,
type, scroll, screenshot — through a bounded, guarded, fully audited
screenshot-step loop. Use it when the system your agent needs to operate has no
clean API: an internal admin panel, a partner portal, a legacy back office.

Endpoint paths below are relative; send them against
`https://api.orbit.devotel.io`. Endpoint schemas are not restated here — see
the [agents endpoint reference](/api-reference/endpoints/agents) for the full
request and response bodies.

<Note>
  There is no dedicated Computer-Use section in the dashboard yet — a
  Computer-Use panel is planned but not shipped. Until it lands, manage tools,
  dry-runs, and session reads through the API surface below.
</Note>

## 1. What a computer-use tool is

An agent's tool belt today composes four tool kinds:

* **Custom tools** — named HTTP endpoints you register once and reference from
  any agent.
* **MCP tools** — capabilities exposed by an external MCP server.
* **A2A** — skills another agent exposes through the A2A federation surface.
* **Computer-use tools** — a bounded browser-operating loop that drives a web
  UI step by step.

For a computer-use tool, each agent turn proposes **one browser action**
(navigate, click, type, …). Orbit evaluates that action against the tool's
guardrails, forwards approved actions to **your sandbox executor**, and writes
an audit record for every step. The sandbox is a tenant-owned executor
endpoint; Orbit never runs a browser itself.

Containment is structural, not policy-level:

* **Bounded step budget** — `max_steps` caps the loop (1–50, default 20). A
  run that exhausts its budget is forced to stop.
* **SSRF-pinned sandbox** — the `sandbox_url` is validated at write time and
  re-pinned at every dispatch, so a tool can never be pointed at internal
  infrastructure.
* **Envelope-encrypted sandbox secret** — `sandbox_secret` is stored
  envelope-encrypted and never returned by any read. Responses expose only a
  `sandbox_secret_set` flag.
* **Audit trail** — every proposed action is recorded, whether it was allowed,
  denied, or gated on confirmation. A run can always be reconstructed later.

## 2. Create a computer-use tool

`POST /api/v1/agents/{agentId}/computer-use-tools` registers a tool on one
agent. Each tool is tenant-scoped and lives under
`/api/v1/agents/{agentId}/computer-use-tools` — full list / update / delete
CRUD alongside create.

Key fields on create:

| Field                  | Required | Purpose                                                                                                                     |
| ---------------------- | -------- | --------------------------------------------------------------------------------------------------------------------------- |
| `name`                 | yes      | Lowercase snake\_case slug (3–64 chars). Built-in platform tool slugs are reserved and rejected.                            |
| `description`          | yes      | What the tool does — shown to the agent when it picks a tool.                                                               |
| `sandbox_url`          | yes      | Your sandbox executor endpoint. Must be a public `https://` URL; loopback, internal suffixes, and private IPs are rejected. |
| `sandbox_secret`       | no       | Shared secret the sandbox uses to verify Orbit's request signature (8–512 chars).                                           |
| `max_steps`            | no       | Loop budget, 1–50 steps. Default 20.                                                                                        |
| `step_timeout_ms`      | no       | Per-action timeout, 1–120 seconds. Default 30 seconds.                                                                      |
| `allowed_domains`      | yes      | Domains the agent may navigate to (1–50 entries). Subdomains of a listed domain are allowed.                                |
| `blocked_actions`      | no       | Action kinds the agent may never perform.                                                                                   |
| `require_confirmation` | no       | Action kinds that gate on an explicit `confirmed` flag from the operator.                                                   |
| `enabled`              | no       | Default true. Disable a tool without deleting it.                                                                           |

The browser action vocabulary is a closed set: `navigate`, `click`,
`double_click`, `right_click`, `type`, `key`, `scroll`, `move`, `drag`,
`wait`, `screenshot`, `back`, `forward`, `refresh`. `navigate` is the only
URL-bearing action — it is the one the domain allowlist gates.

Example:

```bash theme={null}
curl -X POST https://api.orbit.devotel.io/api/v1/agents/agt_7e3b9c1f/computer-use-tools \
  -H "X-API-Key: dv_live_sk_..." \
  -H "Content-Type: application/json" \
  -d '{
    "name": "billing_portal",
    "description": "Drive the internal billing portal to look up and export invoices.",
    "sandbox_url": "https://sandbox.example.com/browser/run",
    "sandbox_secret": "your-sandbox-signing-secret",
    "max_steps": 25,
    "allowed_domains": ["billing.example.com"],
    "blocked_actions": ["type"],
    "require_confirmation": ["navigate"]
  }'
```

The response is `201` with the created tool. Updates go through
`PATCH /api/v1/agents/{agentId}/computer-use-tools/{toolId}` — re-sending
`sandbox_url` re-validates it, sending `sandbox_secret: null` clears the
stored secret, and omitting it leaves the current secret untouched.
`DELETE` removes the tool. A 422 `INVALID_SANDBOX_URL` envelope names why the
sandbox URL failed safety validation (protocol, blocked host, internal
suffix, DNS, or private IP).

Node.js:

```js theme={null}
const res = await fetch(
  `https://api.orbit.devotel.io/api/v1/agents/${agentId}/computer-use-tools`,
  {
    method: 'POST',
    headers: {
      'X-API-Key': process.env.ORBIT_API_KEY,
      'Content-Type': 'application/json',
    },
    body: JSON.stringify({
      name: 'billing_portal',
      description: 'Drive the internal billing portal to look up and export invoices.',
      sandbox_url: 'https://sandbox.example.com/browser/run',
      sandbox_secret: 'your-sandbox-signing-secret',
      max_steps: 25,
      allowed_domains: ['billing.example.com'],
      blocked_actions: ['type'],
      require_confirmation: ['navigate'],
    }),
  },
);
const { data } = await res.json();
console.log(data.tool.id);
```

## 3. Validate guardrails with a dry-run

`POST /api/v1/agents/{agentId}/computer-use-tools/{toolId}/dry-run` evaluates
**one proposed action** against the tool's guardrails before you let an agent
loose on it. Send `{"dispatch": false}` to check the guardrail verdict only,
or `{"dispatch": true}` (the default) to also forward the action to the
sandbox exactly once — marked as a test so the sandbox treats it accordingly.

```bash theme={null}
curl -X POST https://api.orbit.devotel.io/api/v1/agents/agt_7e3b9c1f/computer-use-tools/tool_abc123/dry-run \
  -H "X-API-Key: dv_live_sk_..." \
  -H "Content-Type: application/json" \
  -d '{
    "action": { "kind": "navigate", "url": "https://billing.example.com/invoices" },
    "confirmed": true,
    "dispatch": true
  }'
```

The response carries the guardrail verdict and, when dispatched, the sandbox
result:

* `decision` — `allow`, `deny`, or `require_confirmation`.
* `code` — a machine-readable guardrail code on a non-allow verdict (see
  §5), and `done: true` from a live run when the step budget is exhausted.
* `result` — the sandbox round-trip: HTTP status, duration, response body,
  and the error string when the sandbox rejected the action.

Dry-runs never consume an agent's step budget and are safe to use in CI.

## 4. Audit every run

Every live run is reconstructable from the audit trail. There is no session
table — reads group the per-step action records by run:

* `GET /api/v1/agents/{agentId}/computer-use/sessions` — one row per run:
  status, first/last step timing, step count, and the operator-supplied task.
  Filter with `?status=all|active|completed|failed` (default `all`) and
  `?limit=1–100` (default 25).
* `GET /api/v1/agents/{agentId}/computer-use/sessions/{sessionId}` — one
  run's full ordered action trace: each step's action kind, guardrail
  decision, sandbox status, and duration.

```bash theme={null}
curl "https://api.orbit.devotel.io/api/v1/agents/agt_7e3b9c1f/computer-use/sessions?status=completed&limit=10" \
  -H "X-API-Key: dv_live_sk_..."
```

A session's `status` reflects how the run ended (`completed`, `failed`,
`cancelled`) plus `running` / `needs_input` for runs still in flight.
Step-by-step records are keyed by run, so the trace survives restarts and is
the source of truth the dashboard session list will render.

## 5. Guardrail rules and runtime containment

Each proposed action passes through the tool's guardrails before Orbit
forwards it to the sandbox. The outcome is one of:

| Decision               | Code                    | Meaning                                                                                                    |
| ---------------------- | ----------------------- | ---------------------------------------------------------------------------------------------------------- |
| `deny`                 | `STEP_BUDGET_EXHAUSTED` | The run already consumed `max_steps`; Orbit signals the agent to stop the loop.                            |
| `deny`                 | `ACTION_BLOCKED`        | The action kind is on the tool's `blocked_actions` list.                                                   |
| `deny`                 | `DOMAIN_NOT_ALLOWED`    | A `navigate` target whose host is not on (or a subdomain of) the `allowed_domains` list.                   |
| `require_confirmation` | `CONFIRMATION_REQUIRED` | The action kind is on `require_confirmation`; the step waits for an explicit confirmation before dispatch. |
| `allow`                | —                       | All checks passed; the action is forwarded to the sandbox.                                                 |

At runtime the loop is additionally contained by the
`step_timeout_ms` per-action timeout and by SSRF re-pinning on every single
dispatch — the sandbox URL is re-validated per call, not once at creation.

**Access.** Lists, session reads, and dry-runs require the `agents:read`
scope; create / update / delete require `agents:write` plus the `owner`,
`admin`, or `developer` role. All surfaces are tenant-scoped — an agent id
outside your organization resolves to a 404.

## 6. Where it fits with other tool kinds

Computer-use complements rather than replaces the other tool kinds:

* Reach for a [custom tool](/agents/creating-agents) when the target system
  has an HTTP endpoint you can call.
* Reach for MCP / A2A when the capability is already exposed by a service or
  another agent.
* Reach for computer-use when the only interface is a rendered page — and
  accept the higher per-step cost in exchange for the closed action set,
  step budget, and full audit trail.