Daisy Voice model

How it differs from a deployment

The deployment protocol binds each socket to an agent you configured in the console, and the server runs your tools. Daisy Voice is the opposite: there is no agent. You describe the session over the wire with session.update, and when the model wants to call a tool the runtime hands the call back to you to execute. Same STT → LLM → TTS core, different framing.

Authentication

You get a workspace-scoped key (prefix voa_live_). Keep it server-side. It is shown once at mint time and stored only as a hash.

uv run python backend/scripts/create_daisy_key.py \
  --workspace-slug default --label 'Acme prod'
# → voa_live_xxxxxxxxxxxxxxxxxxxxxxxxxxxx  (store this now)

Send the key on the WebSocket connect. Server-side clients use the Authorization header; browsers (which cannot set WebSocket headers) pass it as the key query parameter instead.

Connecting

Open a WebSocket to wss://<your-voice-orch>/v1/realtime?model=daisy-voice. The runtime closes with a 1008 policy-violation code if the key is missing or invalid, or a 4429 if the per-key concurrent-session limit is reached. On success it emits platform.ready.

const url = new URL('wss://your-voice-orch/v1/realtime');
url.searchParams.set('model', 'daisy-voice');
url.searchParams.set('key', 'voa_live_...'); // browser; server clients use the header instead
const ws = new WebSocket(url.toString());

ws.addEventListener('message', (event) => {
  const e = JSON.parse(event.data);
  console.log(e.type, e);
});

Configuring the session

Send one session.update as your first message, before any audio. It is honored only up front; after the first turn the config is locked (later updates are ignored with a platform.warning). The runtime acks with session.updated. Both the flat shape below and the nested audio.input/audio.output shape are accepted.

ws.addEventListener('open', () => {
  ws.send(JSON.stringify({
    type: 'session.update',
    session: {
      instructions: 'You are a concise, friendly support agent.',
      voice: 'cgSgspJ2msm6clMCkdW9',
      turn_detection: { threshold: 0.5, silence_duration_ms: 300 },
      tools: [
        {
          type: 'function',
          name: 'search_docs',
          description: 'Search the product knowledge base.',
          parameters: {
            type: 'object',
            properties: { query: { type: 'string' } },
            required: ['query'],
          },
        },
      ],
    },
  }));
});

Audio format

Mic audio you send: PCM16, little-endian, 24 kHz, mono, base64-encoded, in input_audio_buffer.append events. TTS audio you receive: same format, in response.output_audio.delta events. Server-side VAD detects turn ends by default; send input_audio_buffer.commit to force a turn boundary.

Tool calling (you run the tools)

Tools are client-executed. When the model decides to call one of the functions you registered, the runtime emits response.function_call_arguments.done and pauses the turn. It does not run anything. You execute the function however you like (your RAG lookup, an HTTP call, an MCP request), then return the result with a conversation.item.create carrying a function_call_output item. The runtime feeds it back to the model and continues to speech. RAG is just this: register a search function and do the retrieval on your side.

If you do not return a result within 30 seconds the runtime feeds the model a timeout error so the assistant can recover in-turn instead of hanging.

ws.addEventListener('message', async (event) => {
  const e = JSON.parse(event.data);
  if (e.type === 'response.function_call_arguments.done') {
    const args = JSON.parse(e.arguments || '{}');
    const output = await runMyTool(e.name, args); // YOUR code: RAG / HTTP / MCP

    ws.send(JSON.stringify({
      type: 'conversation.item.create',
      item: {
        type: 'function_call_output',
        call_id: e.call_id,
        output: JSON.stringify(output), // the output field is a string
      },
    }));
    // The runtime resumes automatically; sending response.create is harmless but optional.
  }
});

Events

The wire events match the deployment protocol for everything except tools. Server → client: platform.ready, conversation.item.input_audio_transcription.completed, response.created, response.output_text.delta/.done, response.output_audio.delta/.done, response.output_audio_transcript.done, response.done, platform.barge_in, platform.usage.turn, and (relay only) response.function_call_arguments.done. Client → server: session.update, input_audio_buffer.append/.commit, conversation.item.create (both message and function_call_output items), response.create, ping.

Session limits

Each session carries platform-default cost ceilings. When one is crossed the runtime closes with a typed code: 4408 (idle timeout or max duration) or 4409 (per-session cost cap). Treat a 1008 close as a rejected session (bad key) and do not auto-reconnect; reconnect only on abnormal 1006 closures.