Skip to main content

Voice Conversations

Start a real-time audio conversation between your user and a Stellar agent. The SDK handles microphone capture, audio playback, and the WebSocket connection.

import { createStellarClient } from "@stellar-ai/agent-sdk";

const client = createStellarClient();

const conversation = await client.startConversation({
  auth: {
    strategy: "publicToken",
    token: "<your-public-access-token>",
  },
  agentId: "<your-agent-id>",
});

Each client supports one active voice conversation at a time. To start a new one, end the current conversation with conversation.end() first, or you'll get a CONVERSATION_ALREADY_ACTIVE error.

In browser or WebView environments, request microphone access before starting:

await navigator.mediaDevices.getUserMedia({ audio: true });

If you're embedding the SDK inside a hybrid app or native shell (Capacitor, Cordova/Ionic, MAUI Blazor Hybrid, Modyo Native App Shell, Tauri, Electron, custom Android WebView or iOS WKWebView, etc.), see Hybrid apps and WebViews for how to grant the WebView mic access from the native side.

Events​

Subscribe to events using .on(). All event methods return the conversation instance for chaining.

conversation
  .on("stateChanged", ({ state }) => {
    console.log("State:", state);
  })
  .on("error", ({ error, code, fatal }) => {
    if (fatal) console.error("Conversation ended:", code, error);
  })
  .on("started", ({ clientId }) => {
    console.log("Started with client ID:", clientId);
  })
  .on("handover", ({ handoverType, phoneNumber, queueId, announcement }) => {
    console.log("Handover:", handoverType);
  })
  .on("actionExecuted", ({ actionName, requestBody, responseBody }) => {
    console.log("Action:", actionName);
  });
EventPayloadDescription
stateChanged{ state }Connection state changed (connecting, connected, disconnected, error)
error{ error, code, fatal }An error occurred; fatal is true when the session ends
started{ clientId, conversationId? }Conversation started successfully
handover{ handoverType, phoneNumber?, queueId?, announcement? }Agent initiated a handover
actionExecuted{ actionName, functionCallItemId?, requestBody?, responseBody?, backgroundExecution? }Agent executed a tool or action
backgroundExecutionProgress{ executionId, actionName, phase, step? }A background action started or updated one of its steps

Handover event​

The handover event fires when the agent initiates a transfer. It uses HandoverType.PHONE for phone transfers and HandoverType.QUEUE for queue-based routing:

import { HandoverType } from "@stellar-ai/agent-sdk";

conversation.on(
  "handover",
  ({ handoverType, phoneNumber, queueId, announcement }) => {
    if (handoverType === HandoverType.PHONE) {
      // Transfer the call to phoneNumber
    } else if (handoverType === HandoverType.QUEUE) {
      // Route to the queue identified by queueId
    }
  },
);

Phone handovers include phoneNumber, queue handovers include queueId, and both may include an announcement.

Action executed event​

The actionExecuted event is the main way to handle Studio Client action executions in your app during a voice conversation. When the agent chooses a client_action, the SDK emits this event with the Studio action name and the arguments the agent supplied. Your app can listen for it and map it to local behavior like navigation, opening native UI, or updating client-side state while the conversation continues.

The same event also fires for other action types. In those cases, requestBody and responseBody let you inspect what happened after the action ran.

For a background action, backgroundExecution contains the terminal outcome, total duration, and an ordered list of step metadata. While it runs, backgroundExecutionProgress reports the started, step_started, and step_completed phases. A progress step includes its id, index, kind, and available status, outcome, duration, and HTTP metadata. Progress updates are transient; use the terminal backgroundExecution on actionExecuted as the completed result.

conversation.on("actionExecuted", ({ actionName, requestBody }) => {
  const args = requestBody ? JSON.parse(requestBody) : {};

  if (actionName === "navigate_to_screen") {
    navigateToScreen(args.screen_name);
  }
});

Methods​

end​

End the conversation and close the connection. After calling end, discard the instance.

await conversation.end();

mute and unmute​

Control the microphone. When muted, the SDK stops capturing audio but keeps the WebSocket connection open.

conversation.mute();
console.log(conversation.isMuted); // true

conversation.unmute();
console.log(conversation.isMuted); // false

isMuted​

A boolean property indicating whether the microphone is currently muted.

state​

The current connection state: connecting, connected, disconnected, or error.

console.log(conversation.state); // "connected"

on, off, and once​

  • on(event, handler) — subscribe to an event. Returns the instance for chaining.
  • off(event, handler) — unsubscribe a handler.
  • once(event, handler) — subscribe for a single occurrence, then auto-unsubscribe.

Next steps​

  • Text chat — messaging-based conversations with streaming responses
  • TypeScript SDK — installation, authentication, and shared configuration