TypeScript SDK

Installation

Install the package in your project through your package manager:

npm install @stellar-ai/agent-sdk
# or
yarn add @stellar-ai/agent-sdk
# or
pnpm install @stellar-ai/agent-sdk

Usage

The Stellar TypeScript SDK is designed for use in TypeScript and JavaScript projects and as a foundation for framework‑specific libraries (e.g. React bindings, widgets).

You can use it in:

• Web apps
• React Native / Expo apps
• Capacitor apps (via the WebView)
• Any JS environment with WebSocket and Web Audio support

Initialize client and conversation

First, create a Stellar client and initialize a conversation:

import { createStellarClient } from "@stellar-ai/agent-sdk";

const client = createStellarClient({
  // Optional: baseUrl defaults to Stellar's production environment URL if not provided
});

const conversation = await client.startConversation({
  auth: {
    strategy: "publicToken",
    token: "<your-public-access-token>", // Get this from agent settings in Stellar dashboard
  },
  agentId: "<your-agent-id>", // Get this from agent settings in Stellar dashboard
});

Note: Each StellarClient instance supports one active conversation at a time. To start a new conversation, first end the current one with conversation.end(), or create a new StellarClient instance. Attempting to start a second conversation without ending the first will throw a CONVERSATION_ALREADY_ACTIVE error.

This will create a new realtime session with your Stellar agent and, depending on configuration, start streaming audio between the SDK and the backend.

In browser or Capacitor WebView environments, you should request microphone access from the user before starting a voice conversation:

// Call after explaining to the user why microphone access is needed
await navigator.mediaDevices.getUserMedia({ audio: true });

Session configuration

The options passed to startConversation control how the session is established and what context is sent to the agent.

Note: The initial connection attempt has a timeout of 10–15 seconds. If the connection cannot be established within this time, startConversation will reject with a TRANSPORT_ERROR.

Agents and authentication

For public agents (embedded on public websites, no user login required), v0.1 uses public access tokens:

• Enable public access for your agent in the Stellar dashboard.
• Copy the public access token and agent ID from the agent settings.
• Pass them to startConversation via the auth and agentId options.

const client = createStellarClient();

const conversation = await client.startConversation({
  auth: {
    strategy: "publicToken",
    token: "<your-public-access-token>", // Get from agent settings
  },
  agentId: "<your-agent-id>", // Get from agent settings
});

Note: Support for private/authenticated agents (where users must log in via your identity provider) is planned for a future release.

Audio configuration

In most cases (including Capacitor apps), you can treat the SDK like a browser client:

• Ask for mic access with getUserMedia.
• Call startConversation() without any special audio options.

The SDK will handle microphone capture via Web Audio and stream it to your Stellar agent, and will automatically play assistant audio via Web Audio. If mic access is required but unavailable or denied, startConversation will throw and/or emit an error event.

Initial context and variables

You can pass initial context to the agent using variables in startConversation:

const conversation = await client.startConversation({
  auth: {
    strategy: "publicToken",
    token: "<your-public-access-token>",
  },
  agentId: "<your-agent-id>",
  variables: {
    userName: "Alice",
    orderId: "12345",
    isPremium: true,
  },
});

Variables are key-value pairs (strings, numbers, booleans, or null) that provide context to the agent at the start of the conversation. These can be used by the agent's system prompt or tools to personalize the interaction.

Events

The SDK is event‑driven. You can subscribe to conversation events using the .on method:

• stateChanged – fired when the connection state changes (connecting, connected, disconnected, error).
• error – fired when an error occurs.
• started – fired when the conversation is successfully started. Contains a clientId (validated client identifier) that can be used for session tracking and analytics.
• handover – fired when the agent initiates a handover to another system or human agent.
• actionExecuted – fired when the agent executes a tool or action. Contains the actionName and optionally the requestBody and responseBody.

Example:

const conversation = await client.startConversation({
  auth: { strategy: "publicToken", token: "<your-token>" },
  agentId: "<your-agent-id>",
});

conversation
  .on("stateChanged", ({ state }) => {
    console.log("Conversation state:", state);
  })
  .on("error", ({ error, code }) => {
    console.error("Conversation error", code, error);
  })
  .on("started", ({ clientId }) => {
    console.log("Conversation started with client ID:", clientId);
  })
  .on("handover", ({ handoverType, phoneNumber, queueId, announcement }) => {
    console.log("Handover initiated:", handoverType);
  })
  .on("actionExecuted", ({ actionName, requestBody, responseBody }) => {
    console.log("Action executed:", actionName);
  });

Handover event

The handover event is emitted when the agent initiates a handover. The event payload contains:

• handoverType – the type of handover being performed.
• phoneNumber (optional) – a phone number to transfer the call to.
• queueId (optional) – the queue ID to route the call to.
• announcement (optional) – a message to announce during the handover.

Action executed event

The actionExecuted event is emitted when the agent executes a tool or action during the conversation. The event payload contains:

• actionName – the name of the action that was executed.
• requestBody (optional) – the request body sent to the action.
• responseBody (optional) – the response body returned by the action.

Return value: Conversation instance

startConversation returns a Conversation instance that can be used to control the session. The method will throw if the session cannot be established (for example, if auth fails or microphone access is required and denied).

The main methods are:

end

Manually end the conversation and close the underlying connection. After calling end, the Conversation instance should be considered finished and can be discarded.

await conversation.end();

mute and unmute

Control the microphone. When muted, the SDK stops capturing audio from the microphone. The WebSocket connection remains open.

// Mute the microphone
conversation.mute();
console.log(conversation.isMuted); // true

// Unmute the microphone
conversation.unmute();
console.log(conversation.isMuted); // false

isMuted

A boolean property indicating whether the microphone is currently muted.

if (conversation.isMuted) {
  console.log("Microphone is muted");
}

state

The current connection state of the conversation. Possible values:

• connecting – establishing connection to Stellar
• connected – active conversation in progress
• disconnected – conversation ended normally
• error – connection failed or encountered an error

console.log(conversation.state); // "connected"

on, off, and once

Event subscription methods for listening to conversation events:

• on(event, handler) – Subscribe to an event. Returns the Conversation instance for chaining.
• off(event, handler) – Unsubscribe a specific handler from an event. Returns the Conversation instance for chaining.
• once(event, handler) – Subscribe to an event for a single occurrence, then automatically unsubscribe. Returns the Conversation instance for chaining.

const handler = ({ state }) => console.log("State:", state);

// Subscribe
conversation.on("stateChanged", handler);

// Unsubscribe
conversation.off("stateChanged", handler);

// Listen once
conversation.once("stateChanged", ({ state }) => {
  console.log("First state change:", state);
});

React Native / Expo

The SDK ships with ready-made audio implementations for Expo via the @stellar-ai/agent-sdk/expo subpath. These wrap @mykin-ai/expo-audio-stream and handle sample-rate conversion automatically.

npx expo install @mykin-ai/expo-audio-stream

import { createStellarClient } from "@stellar-ai/agent-sdk";
import {
  ExpoAudioCapture,
  ExpoAudioPlayback,
} from "@stellar-ai/agent-sdk/expo";

const client = createStellarClient({
  audioCapture: new ExpoAudioCapture(),
  audioPlayback: new ExpoAudioPlayback(),
});

Custom audio

The SDK uses the Web Audio API by default (browsers). For other environments, you can provide custom implementations of IAudioCapture and IAudioPlayback:

import { createStellarClient } from "@stellar-ai/agent-sdk";
import type { IAudioCapture, IAudioPlayback } from "@stellar-ai/agent-sdk";

const client = createStellarClient({
  audioCapture: new MyCustomAudioCapture(),
  audioPlayback: new MyCustomAudioPlayback(),
});

IAudioCapture

Method / Property	Description
start(onAudioData: (data: Int16Array) => void)	Start capturing. Call the callback with PCM16 24kHz mono chunks.
stop()	Stop capturing and release resources.
mute()	Mute the microphone.
unmute()	Unmute the microphone.
muted (getter)	Whether the microphone is currently muted.

IAudioPlayback

Method	Description
start()	Initialize the audio output.
stop()	Stop playback and release resources.
play(pcm16Data: Int16Array)	Queue a PCM16 24kHz mono chunk for playback.
interrupt()	Stop all current playback immediately (used for barge-in).

Error handling

Errors can surface in two ways:

• As rejected promises (e.g. startConversation).
• As error events emitted by the Conversation instance.

Error codes:

• UNAUTHENTICATED: Authentication failures (e.g. expired or invalid token).
• TRANSPORT_ERROR: Network issues (e.g. WebSocket cannot connect, connection timeout).
• MISSING_AGENT_ID: The agentId was not provided in startConversation options.
• MIC_ACCESS_DENIED: Microphone access denied by the user.
• CONVERSATION_ALREADY_ACTIVE: A conversation is already in progress on this client instance. End it with conversation.end() before starting a new one.
• INTERNAL_ERROR: Unexpected SDK errors.

Example:

try {
  const conversation = await client.startConversation({
    auth: { strategy: "publicToken", token: "<your-token>" },
    agentId: "<your-agent-id>",
  });
  conversation.on("error", ({ error, code }) => {
    console.error("Conversation error", code, error);
  });
} catch (err) {
  console.error("Failed to start conversation", err);
}

Resource cleanup and lifecycle

The SDK automatically cleans up resources (closes WebSocket connections, releases microphone access) when the page unloads or the app is closed. In most cases, you don't need to manually call conversation.end() unless you want to explicitly end a conversation before the page unloads.

Capacitor-specific notes

• In a Capacitor app, the SDK uses Web Audio for microphone capture through the WebView.
• The SDK does not manage app backgrounding; your app should:
  • End conversations when going to background if appropriate.
  • Optionally start new conversations on resume.

Environment requirements

• Browser: Modern browser with WebSocket and Web Audio support, HTTPS in production (except localhost).
• React Native / Expo: Install @mykin-ai/expo-audio-stream and use the built-in Expo implementations (see React Native / Expo).

Versioning

The TypeScript SDK follows semantic versioning:

• Initial release: 0.1.0.
• New functionality that does not break existing APIs will increment the minor version.
• Breaking changes to the public API surface (client options, conversation methods, event names) will increment the major version and be documented with migration steps.