TypeScript SDK

The Stellar TypeScript SDK lets you embed real-time voice and text chat conversations with AI agents in your application.

Installation

npm install @stellar-ai/agent-sdk
# or
yarn add @stellar-ai/agent-sdk
# or
pnpm install @stellar-ai/agent-sdk

The SDK works in:

• Web apps
• React Native / Expo apps
• Hybrid apps that render their UI in an embedded WebView — Capacitor, Cordova/Ionic, Tauri, Electron, MAUI Blazor Hybrid, Modyo Native App Shell, or any custom Android WebView / iOS WKWebView shell (see Hybrid apps and WebViews)
• Any JS environment with WebSocket support (voice conversations additionally require Web Audio or a custom audio implementation)

Client setup

Create a client to start conversations:

import { createStellarClient } from "@stellar-ai/agent-sdk";

const client = createStellarClient();

Options:

• baseUrl — optional base URL (defaults to Production)
• logger — optional logger for debugging (with debug, warn, error methods)
• audioCapture — optional custom audio capture implementation (see Custom audio)
• audioPlayback — optional custom audio playback implementation (see Custom audio)

From the client you can start a voice conversation with client.startConversation() or a text chat with client.startChatConversation().

Authentication

For public agents (embedded on websites, no user login required), use public access tokens:

1. Enable public access for your agent in the Stellar dashboard.
2. Copy the public access token and agent ID from the agent settings. Each environment (Development, Staging, Production) has its own token — use the environment selector in the Sharing tab to pick the right one.
3. Pass them when starting a conversation:

const conversation = await client.startConversation({
  auth: {
    strategy: "publicToken",
    token: "<your-public-access-token>",
  },
  agentId: "<your-agent-id>",
});

info

Support for private/authenticated agents (where users log in via your identity provider) is available on request. Contact us to discuss your requirements.

Initial context and variables

Pass key-value pairs to provide context to the agent at the start of any conversation. These can be used by the agent's system prompt or tools to personalize the interaction.

const conversation = await client.startConversation({
  auth: { strategy: "publicToken", token: "<your-token>" },
  agentId: "<your-agent-id>",
  variables: {
    userName: "Alice",
    orderId: "12345",
    isPremium: true,
  },
});

Variables work the same way for both voice and chat conversations.

Error handling

Errors can surface in two ways:

• As rejected promises from startConversation or startChatConversation.
• As error events emitted by the conversation instance.

Code	Description
UNAUTHENTICATED	Authentication failed (expired or invalid token)
TRANSPORT_ERROR	Network issues (WebSocket connection failed, timeout)
MISSING_AGENT_ID	The agentId was not provided
MIC_ACCESS_DENIED	Microphone access denied by the user
CONVERSATION_ALREADY_ACTIVE	A conversation is already in progress — end it first
INTERNAL_ERROR	Unexpected SDK error

try {
  const conversation = await client.startConversation({
    auth: { strategy: "publicToken", token: "<your-token>" },
    agentId: "<your-agent-id>",
  });
  conversation.on("error", ({ error, code }) => {
    console.error("Conversation error:", code, error);
  });
} catch (err) {
  console.error("Failed to start conversation:", err);
}

The initial connection attempt has a timeout of 15 seconds. If the connection cannot be established, the method rejects with a TRANSPORT_ERROR.

React Native / Expo

The SDK ships with ready-made audio implementations for Expo via the @stellar-ai/agent-sdk/expo subpath. These wrap @mykin-ai/expo-audio-stream and handle sample-rate conversion automatically.

npx expo install @mykin-ai/expo-audio-stream

import { createStellarClient } from "@stellar-ai/agent-sdk";
import {
  ExpoAudioCapture,
  ExpoAudioPlayback,
} from "@stellar-ai/agent-sdk/expo";

const client = createStellarClient({
  audioCapture: new ExpoAudioCapture(),
  audioPlayback: new ExpoAudioPlayback(),
});

Custom audio

The SDK uses the Web Audio API by default in browsers. For other environments, you can provide custom implementations of IAudioCapture and IAudioPlayback:

import { createStellarClient } from "@stellar-ai/agent-sdk";
import type { IAudioCapture, IAudioPlayback } from "@stellar-ai/agent-sdk";

const client = createStellarClient({
  audioCapture: new MyCustomAudioCapture(),
  audioPlayback: new MyCustomAudioPlayback(),
});

IAudioCapture

Method / Property	Description
start(onAudioData: (data: Int16Array) => void)	Start capturing. Call the callback with PCM16 24kHz mono chunks.
stop()	Stop capturing and release resources.
mute()	Mute the microphone.
unmute()	Unmute the microphone.
muted (getter)	Whether the microphone is currently muted.

IAudioPlayback

Method	Description
start()	Initialize the audio output.
stop()	Stop playback and release resources.
play(pcm16Data: Int16Array)	Queue a PCM16 24kHz mono chunk for playback.
interrupt()	Stop all current playback immediately (used for barge-in).

Hybrid apps and WebViews

The SDK runs unchanged inside any embedded WebView — Capacitor, Cordova/Ionic, Tauri, Electron, MAUI Blazor Hybrid, Modyo Native App Shell, or a custom Android WebView / iOS WKWebView. WebSocket connections, dynamic actions, and event listeners (actionExecuted, error, transcript events, etc.) all behave exactly as they do in a regular browser. If your native side needs to react to in-conversation events, forward them through your platform's JS-to-native bridge (e.g. Capacitor plugins, Blazor's [JSInvokable] methods invoked via DotNet.invokeMethodAsync, WKScriptMessageHandler on iOS, addJavascriptInterface on Android).

The one area that needs native-shell wiring is microphone access. Voice conversations call getUserMedia inside the WebView, and most embedded browsers will deny that request unless the host app has been configured correctly.

Microphone permission passthrough

For voice conversations to work, the WebView's getUserMedia call must succeed. That requires three things, and all of them live in the native shell — not the JS code:

1. Declare the OS-level permission in the native app manifest.
   • iOS: add NSMicrophoneUsageDescription to Info.plist with a user-facing reason.
   • Android: add <uses-permission android:name="android.permission.RECORD_AUDIO" /> to AndroidManifest.xml.
2. Request the permission at runtime before the user starts a conversation. iOS prompts automatically the first time the mic is used; Android 6+ requires an explicit runtime request.
3. Grant the permission through to the WebView:
   • Android WebView: set a WebChromeClient and override onPermissionRequest(PermissionRequest), then call request.grant(request.getResources()) when the resources include PermissionRequest.RESOURCE_AUDIO_CAPTURE. Serve your page over HTTPS — Android WebView only exposes getUserMedia to secure origins. Some Android setups also need MODIFY_AUDIO_SETTINGS declared in the manifest alongside RECORD_AUDIO.
   • iOS WKWebView: assign a WKUIDelegate and implement webView(_:requestMediaCapturePermissionFor:initiatedByFrame:type:decisionHandler:) (iOS 15+), checking for WKMediaCaptureType.microphone and calling decisionHandler(.grant).
   • MAUI Blazor Hybrid: customize the underlying BlazorWebView handler — on Android, attach a WebChromeClient that overrides OnPermissionRequest; on iOS/Mac Catalyst, set a WKUIDelegate that implements the media-capture permission method above.
   • Capacitor / Cordova: install the standard microphone permission plugin and follow its setup.
   • Tauri / Electron: declare microphone access in your Tauri capabilities (and on Android, add RECORD_AUDIO / MODIFY_AUDIO_SETTINGS to the manifest) or, in Electron, handle session.setPermissionRequestHandler for the media permission.

If getUserMedia succeeds, the SDK works without any further changes. If it rejects, the SDK emits a MIC_ACCESS_DENIED error.

When WebView audio isn't viable

Some environments make WebView mic capture impractical: older Android/iOS versions with limited getUserMedia support, MDM-managed devices that block WebView media, kiosk hardware, or cases where you need native-grade audio control (echo cancellation tuning, custom DSP, multi-channel input).

For those cases, capture audio natively and feed it into the SDK via the custom audioCapture hook. Your native code captures PCM16 mono at 24 kHz, hands the chunks across the JS bridge, and the SDK's IAudioCapture.start callback forwards them on the wire. This bypasses the WebView audio stack entirely.

Resource cleanup and lifecycle

The SDK automatically cleans up resources (WebSocket connections, microphone access) when the page unloads or the app closes. Call conversation.end() explicitly if you want to end a conversation before that.

Inside a hybrid app or WebView, the SDK uses Web Audio for microphone capture through the embedded browser (see Hybrid apps and WebViews). The SDK doesn't manage app backgrounding — your app should end conversations when going to background if appropriate, and optionally start new ones on resume.

Environment requirements

• Browser: Modern browser with WebSocket and Web Audio support, HTTPS in production (except localhost).
• React Native / Expo: Install @mykin-ai/expo-audio-stream and use the built-in Expo implementations (see React Native / Expo).
• Node.js: 18+

Next steps

• Voice conversations — real-time audio conversations with agents
• Text chat — messaging-based conversations with streaming responses