> ## Documentation Index
> Fetch the complete documentation index at: https://docs.orbit.devotel.io/llms.txt
> Use this file to discover all available pages before exploring further.

# Dry-run an agent against a scripted scenario

> Replay a scripted multi-turn scenario against the agent in sandbox mode and return the turn-by-turn trace plus assertion results. Live tool side effects (calendar, transfer, mutative memory) are short-circuited and billing is skipped, so the LLM call is real but nothing is committed. Use it to test prompt changes and tool behaviour before deploying. Requires owner, admin, or developer.



## OpenAPI

````yaml /openapi.yaml post /api/v1/agents/{id}/dry-run
openapi: 3.1.0
info:
  title: Devotel CPaaS API
  description: Orbit by Devotel — Communications Platform as a Service API
  version: 1.0.0
  contact:
    name: Devotel
    url: https://devotel.io
    email: support@devotel.io
  license:
    name: Proprietary
servers:
  - url: https://api.orbit.devotel.io
    description: Production
security:
  - Bearer: []
  - ApiKey: []
tags:
  - name: Messages
    description: >-
      Send and manage messages across all channels (SMS, WhatsApp, RCS, Email,
      Viber, etc.)
  - name: Fax
    description: >-
      List and track fax (MMS/T.38) transmissions on Telnyx-backed fax numbers
      (sending flows through the Messaging API)
  - name: Agents
    description: AI agent creation, configuration, and execution
  - name: Voice
    description: Voice calls, IVR, conferencing, and SIP trunking
  - name: OnCall
    description: On-call rotations and escalation policy planning for incident alerting
  - name: Webhooks
    description: Webhook endpoint management and delivery logs
  - name: Numbers
    description: Phone number search, provisioning, and configuration
  - name: Brand Identity
    description: >-
      Unified cross-channel brand trust posture (10DLC, toll-free, WhatsApp,
      RCS, branded calling, number KYC)
  - name: Contacts
    description: Contact management, segmentation, and lifecycle tracking
  - name: Campaigns
    description: Marketing campaign orchestration and analytics
  - name: Flows
    description: Automation flow builder and execution engine
  - name: Templates
    description: Message template management and approval workflows
  - name: Settings
    description: Organization, channel, and user preference settings
  - name: Verify
    description: OTP generation and verification across channels
  - name: Push
    description: Push notification delivery via FCM and APNs
  - name: Integrations
    description: Third-party service connections and OAuth management
  - name: CDP
    description: >-
      Customer Data Platform — activation surface (CRM object sync, streaming
      destinations, ad-audience activation)
  - name: Files
    description: >-
      Server-to-server media upload, listing, retrieval, and deletion
      (signed-URL backed)
  - name: Commerce
    description: >-
      Omnichannel conversational-commerce — persistent cart + checkout state
      machine, channel-agnostic hosted pay-by-link, native WhatsApp checkout,
      and AP2-style agent payment mandates.
  - name: Sync
    description: >-
      Real-time shared-state primitive (Twilio Sync parity) — Documents, Maps,
      Lists, and ephemeral Streams, with change events relayed over the
      /api/v1/ws/sync WebSocket gateway
paths:
  /api/v1/agents/{id}/dry-run:
    post:
      tags:
        - Agents
      summary: Dry-run an agent against a scripted scenario
      description: >-
        Replay a scripted multi-turn scenario against the agent in sandbox mode
        and return the turn-by-turn trace plus assertion results. Live tool side
        effects (calendar, transfer, mutative memory) are short-circuited and
        billing is skipped, so the LLM call is real but nothing is committed.
        Use it to test prompt changes and tool behaviour before deploying.
        Requires owner, admin, or developer.
      parameters:
        - schema:
            type: string
          in: path
          name: id
          required: true
          description: Agent identifier (Devotel agent_xxx id).
      requestBody:
        required: true
        content:
          application/json:
            schema:
              type: object
              required:
                - scenario
              properties:
                scenario:
                  type: object
                  required:
                    - turns
                  properties:
                    turns:
                      type: array
                      minItems: 1
                      maxItems: 20
                      items:
                        type: object
                        required:
                          - user_msg
                        properties:
                          user_msg:
                            type: string
                            minLength: 1
                            maxLength: 8000
                            description: Simulated user message for this turn.
                    expected_tool_calls:
                      type: array
                      items:
                        type: string
                      description: >-
                        Tool names that must appear at least once across the
                        trace.
                    must_contain:
                      type: array
                      items:
                        type: string
                      description: >-
                        Substrings that must appear in the assistant transcript
                        (case-insensitive).
                    must_not_contain:
                      type: array
                      items:
                        type: string
                      description: >-
                        Substrings that must NOT appear in the assistant
                        transcript.
                mock_tools:
                  type: boolean
                  default: true
                  description: Mock mutative tools during the run. Defaults to true.
      responses:
        '200':
          description: Dry-run trace and assertion verdict
          content:
            application/json:
              schema:
                type: object
                description: Dry-run trace and assertion verdict
                properties:
                  data:
                    type: object
                    properties:
                      agent_id:
                        type: string
                      mock_tools:
                        type: boolean
                      status:
                        type: string
                        enum:
                          - pass
                          - fail
                        description: Overall verdict across all turns and assertions.
                      duration_ms:
                        type: integer
                      trace:
                        type: array
                        items:
                          type: object
                          properties:
                            index:
                              type: integer
                            user_msg:
                              type: string
                            assistant_response:
                              type: string
                            tool_calls:
                              type: array
                              items:
                                type: object
                                additionalProperties: true
                            latency_ms:
                              type: integer
                            tokens_used:
                              type: integer
                            error:
                              type:
                                - 'null'
                                - string
                              description: Present only when this turn failed.
                      assertions:
                        type: array
                        items:
                          type: object
                          properties:
                            name:
                              type: string
                            status:
                              type: string
                              enum:
                                - pass
                                - fail
                            reason:
                              type:
                                - 'null'
                                - string
                  meta:
                    type: object
                    properties:
                      request_id:
                        type: string
                      timestamp:
                        type: string
                        format: date-time
components:
  securitySchemes:
    Bearer:
      type: http
      scheme: bearer
      bearerFormat: JWT
      description: Dashboard JWT token from Clerk
    ApiKey:
      type: apiKey
      name: X-API-Key
      in: header
      description: Server-to-server API key (dv_live_sk_*)

````