Skip to main content

Voice Conversations

Start a real-time audio conversation between your user and a Stellar agent. The SDK handles microphone capture, audio playback, and the WebSocket connection.

import { createStellarClient } from "@stellar-ai/agent-sdk";

const client = createStellarClient();

const conversation = await client.startConversation({
  auth: {
    strategy: "publicToken",
    token: "<your-public-access-token>",
  },
  agentId: "<your-agent-id>",
});

Each client supports one active voice conversation at a time. To start a new one, end the current conversation with conversation.end() first, or you'll get a CONVERSATION_ALREADY_ACTIVE error.

In browser or Capacitor WebView environments, request microphone access before starting:

await navigator.mediaDevices.getUserMedia({ audio: true });

Events​

Subscribe to events using .on(). All event methods return the conversation instance for chaining.

conversation
  .on("stateChanged", ({ state }) => {
    console.log("State:", state);
  })
  .on("error", ({ error, code }) => {
    console.error("Error:", code, error);
  })
  .on("started", ({ clientId }) => {
    console.log("Started with client ID:", clientId);
  })
  .on("handover", ({ handoverType, phoneNumber, queueId, announcement }) => {
    console.log("Handover:", handoverType);
  })
  .on("actionExecuted", ({ actionName, requestBody, responseBody }) => {
    console.log("Action:", actionName);
  });
EventPayloadDescription
stateChanged{ state }Connection state changed (connecting, connected, disconnected, error)
error{ error, code }An error occurred
started{ clientId, conversationId? }Conversation started successfully
handover{ handoverType, phoneNumber?, queueId?, announcement? }Agent initiated a handover
actionExecuted{ actionName, requestBody?, responseBody? }Agent executed a tool or action

Handover event​

The handover event fires when the agent initiates a transfer. It uses HandoverType.PHONE for phone transfers and HandoverType.QUEUE for queue-based routing:

import { HandoverType } from "@stellar-ai/agent-sdk";

conversation.on("handover", ({ handoverType, phoneNumber, queueId, announcement }) => {
  if (handoverType === HandoverType.PHONE) {
    // Transfer the call to phoneNumber
  } else if (handoverType === HandoverType.QUEUE) {
    // Route to the queue identified by queueId
  }
});

Phone handovers include phoneNumber, queue handovers include queueId, and both may include an announcement.

Action executed event​

The actionExecuted event fires when the agent runs a tool or action during the conversation. You can use this to update your UI in response to agent actions — for example, displaying order details the agent just looked up.

conversation.on("actionExecuted", ({ actionName, requestBody, responseBody }) => {
  if (actionName === "lookupOrder") {
    const order = JSON.parse(responseBody);
    showOrderDetails(order);
  }
});

Methods​

end​

End the conversation and close the connection. After calling end, discard the instance.

await conversation.end();

mute and unmute​

Control the microphone. When muted, the SDK stops capturing audio but keeps the WebSocket connection open.

conversation.mute();
console.log(conversation.isMuted); // true

conversation.unmute();
console.log(conversation.isMuted); // false

isMuted​

A boolean property indicating whether the microphone is currently muted.

state​

The current connection state: connecting, connected, disconnected, or error.

console.log(conversation.state); // "connected"

on, off, and once​

  • on(event, handler) — subscribe to an event. Returns the instance for chaining.
  • off(event, handler) — unsubscribe a handler.
  • once(event, handler) — subscribe for a single occurrence, then auto-unsubscribe.

Next steps​

  • Text chat — messaging-based conversations with streaming responses
  • TypeScript SDK — installation, authentication, and shared configuration