Start an eval run against a dataset
Kick off an asynchronous eval run: the LLM judge scores the agent against every row of dataset_id under a rubric (correctness, helpfulness, safety, groundedness, or custom — the last requires rubric_prompt). Optionally pin judge_model and a pass threshold (0-100, default 70). Returns immediately with status: pending; poll the run detail for results. Requires the agents:write scope.
Authorizations
Dashboard JWT token from Clerk
Headers
Stripe-style idempotency token. Pass a stable, client-generated value (1-255 chars) to dedupe retries on transient timeouts. The same key+credential+path replays the original response for 24h on 2xx (5min on 4xx, 30s on 5xx). Returns 409 if a concurrent request with the same key is already in flight; replayed responses include the Idempotency-Replay: true response header.
1 - 255Sandbox opt-in for Clerk-session-authenticated requests. Set to true to route the call through the test-mode pipeline: no real provider delivery, no credits deducted, response meta.test_mode: true. Ignored for live API keys (dv_live_sk_*) — server-to-server clients must use a test-prefixed key (dv_test_sk_*) to exercise sandbox. Test-prefixed keys unconditionally enable sandbox regardless of this header.
true, false Path Parameters
Response
Successful response.