Skip to main content

​
Practice Studio: stateless simulated-customer roleplay sessions

Practice Studio trains human agents against a simulated customer. An agent working in the dashboard picks a scenario — for example, a frustrated subscriber disputing a duplicate charge — and an LLM plays the customer while the agent rehearses the conversation turn by turn. When the session ends, a separate call grades it against the scenario’s rubric and returns per-dimension coaching feedback. This page explains the model: how the API stays stateless, how a scenario resolves, and where the surface fits next to QA evaluations. For the step-by-step dashboard and curl walkthrough, see the Practice Studio roleplay guide. Everything a session produces is synthetic — no real customer is contacted, and no real messages, calls, or charges result.

​
1. Stateless by contract: the client replays the transcript

Practice Studio has no server-side session object. There is no session id to create, no stored transcript to append to, and nothing to fetch or close. Instead, every request carries the full running transcript, and the caller rebuilds it between turns:
  • The agent sends a turn: POST /api/v1/practice-studio/turn with the scenario and the transcript so far (customer lines and agent lines, oldest first).
  • The response is the simulated customer’s next line, plus session signals (mood, satisfaction, resolved, ended) — signals this build computes purely from the input it was given.
  • The client appends that line to its local transcript. On the next turn it replays the longer transcript; the API needs no memory of the earlier calls.
That is the entire model. Two consequences follow directly, and both are features rather than limitations:
  • Any client can hold the state. The dashboard widget, a mobile surface, and a curl loop can all run the same session, pass it between each other, or resume it hours later from a stored transcript — because the state is the transcript, not a row that only one server process knows about.
  • Nothing persists, ever. A practice transcript never lands in a tenant table, so roleplay content (which can contain deliberately difficult, synthetic customer text) never mixes with real interaction data, and a session carries no retention question. The endpoints do accept the tenant-scoped auth context, but only to attribute LLM usage to your workspace — never to read or write training state.
The same transcript shape powers both endpoints: each entry is { "role": "customer" | "agent", "content": "<text>" }, oldest first. On the very first turn the array may be empty — the customer speaks first, so an empty transcript returns the scenario’s opening line.

​
2. Scenario resolution: exactly one of preset or inline

Every turn and score request identifies its scenario in one of two mutually exclusive ways, enforced by the request schema (supply both or neither and the API returns 422 VALIDATION_ERROR):
  1. Preset by id — scenarioId references one of the built-in scenarios. GET /api/v1/practice-studio/scenarios returns the catalog: each entry has a stable id, title, channel (chat, voice, or email framing), difficulty, a persona, the situation the customer brings, the objective the trainee must accomplish, and the weighted rubric they will be graded against.
  2. Inline custom — scenario carries a complete scenario object in the request itself: same shape, validated to the same contract (persona with name and mood, bounded situation/objective lengths, a rubric of 1–12 weighted criteria). The scenario lives only in your requests — nothing is stored — so supervisors curating exercises beyond the built-ins own their scenario definitions client-side.
When scenarioId does not match a preset, the API returns 404 NOT_FOUND — a stale id is a lookup failure, not a validation error. One deliberate asymmetry governs the catalog: GET /scenarios serves presets without their hiddenContext — the private facts the simulated customer reveals only when the trainee asks the right questions. Shipping the answer key in the list endpoint would spoil the discovery the exercise is designed to reward. The full scenario, hidden context included, is only ever used server-side to drive the simulation.

​
3. The turn loop

POST /api/v1/practice-studio/turn answers with the customer’s next message and nothing else — plus four signals the client uses to run the session:
  • mood — the customer’s emotional register after the agent’s most recent turn (neutral through angry).
  • satisfaction — a 0–1 reading of how the interaction is going.
  • resolved — the customer considers their issue handled.
  • ended — the customer would naturally end the conversation now (resolved, or they have given up).
The simulated customer reacts to how well the trainee handles it: it warms up when the agent is helpful and stays difficult when they are dismissive, and harder difficulty bands make it less forthcoming. It never breaks character and never coaches the trainee — grading is the score endpoint’s job, not the customer’s. When the client disconnects mid-turn, the upstream generation is cancelled rather than left running.

​
4. Scoring: grade the finished session against its rubric

When the transcript is complete, POST /api/v1/practice-studio/score grades it. Unlike the turn endpoint, scoring requires at least one transcript entry (422 on an empty list) — a session with no turns has nothing to grade. The result is:
  • overallScore — a weighted 0–100, computed server-side from the rubric’s weights. The LLM judge proposes the per-criterion scores; the platform combines the weights itself and never trusts the model’s arithmetic, so the same transcript and rubric always land on the same overall.
  • criteria — one entry per rubric criterion with a 0–100 score and one-to-two sentences of feedback keyed to what the trainee actually wrote.
  • strengths and improvements — the specific things that landed and the concrete gaps to coach.
  • summary — a one-paragraph coaching note.
  • objectiveMet — whether the trainee accomplished the scenario’s stated objective.
Only the trainee’s own turns are graded — what the simulated customer said never inflates or deflates the score.

​
Worked turn-and-score pair

Advance one turn (client-replayed transcript, scenario by preset id):
The response is the next customer line plus the session signals:
Then grade the finished session — the same scenario reference, the full transcript:

​
5. Rate limits and LLM gating

Three controls bound the surface, and all are tenant-owned (configured by the platform, applied to your workspace; none gate on geography or channel):
  • Catalog reads — GET /scenarios rides the standard authenticated-read bucket (higher cap, since dashboards poll lists aggressively).
  • Generation calls — POST /turn and POST /score share the tighter agent-invoke bucket, because every call spends real Anthropic tokens. The cap applies per workspace; the LLM owns real spend, so training traffic is kept away from an org-wide free-for-all.
  • LLM availability — both generation endpoints check that the Anthropic key is configured before spending anything, and reject with 503 LLM_UNAVAILABLE when it is not. A malformed body gets 422 VALIDATION_ERROR; an unknown preset id gets 404 NOT_FOUND; an upstream model failure surfaces as 502 AI_ERROR with failure logged rather than a crash.
Your own tenant-level controls — API keys, dashboard role scope, and the LLM spend accounting a turn or score is attributed to — live in the authentication and billing surfaces; this surface itself adds no extra tenancy rules.

​
6. Where it fits: next to QA evaluations, not inside them

Practice Studio and the QA evaluations and leaderboard model are the two halves of one coaching loop, at opposite ends of the conversation:
  • Practice Studio trains before real calls. Rehearse the hard conversation, leave with rubric-keyed coaching, risk nothing.
  • QA evaluations grade after. Scorecards, the acknowledge/appeal lifecycle, and the leaderboard measure the calls that actually happened.
The two are deliberately disconnected in data: a practice session never counts toward an agent’s evaluation record, the acknowledge/appeal lifecycle, or the leaderboard. A bad practice run carries no risk — that separation is what makes it a safe rehearsal space. Run both together: assign a scenario, then confirm the skill transfers by watching the agent’s real-call QA scores against the same competencies.

​
7. What it is not

  • Not the AI agent architecture. The AI agent architecture page describes the platform’s autonomous agents — the ones that answer real customers across flows, retrieval, and tools. Practice Studio’s LLM plays only the customer, in a sandbox, for training. It routes no customer traffic, holds no conversations with real contacts, and its sessions create no runs, tool calls, or grounding citations.
  • Not a QA score. No evaluation row, appeal, or leaderboard point ever comes out of it (see above).
  • Not a persisted feature. There is no session resource to list, resume server-side, or delete by design — the transcript the client replays is the whole state.

​
See also