@molecule/api-ai

Model-agnostic AI chat interface for molecule.dev.

Defines the AIProvider interface that bond packages (Anthropic, OpenAI, etc.) implement, plus types for messages, streaming events, tool use, and token usage.

Auto-generated, AI-first package reference for the molecule.dev ecosystem. It is written to be read by coding agents as much as by people, and is generated from this package's source — edit src/index.ts JSDoc, not this file.

Quick Start

import { requireProvider } from '@molecule/api-ai'
import type { ChatParams } from '@molecule/api-ai'

const ai = requireProvider()
const params: ChatParams = {
  messages: [{ role: 'user', content: 'Hello!' }],
  stream: true,
}
let reply = ''
for await (const event of ai.chat(params)) {
  if (event.type === 'text') reply += event.content // or forward the chunk to the client (SSE)
}
console.log(reply)

Type

core

Installation

npm install @molecule/api-ai @molecule/api-bond @molecule/api-i18n

API

Interfaces

ChatMessage

Chat message in a conversation.

interface ChatMessage {
  role: 'user' | 'assistant' | 'system'
  content: string | ContentBlock[]
  /**
   * Model-native reasoning text produced alongside this (assistant) message —
   * e.g. an OpenAI-compatible provider's `reasoning_content`. Some providers
   * (Moonshot's Kimi K3 / k2.7-code and other preserved-thinking models)
   * require it to be replayed verbatim on subsequent requests within a
   * tool-call loop to keep reasoning continuity; callers that captured
   * `thinking` stream events should set it when rebuilding history. Bonds
   * whose provider has no such requirement ignore it.
   */
  reasoning?: string
}

ProviderResponseMeta

What the provider said about the call itself — the ids an investigation needs when a reply is wrong.

A completion that does not belong to the request that asked for it is not hypothetical: on 2026-10-16 a molecule.dev conversation was handed a complete foreign answer (a Flutter/.NET ticket review) on a stream whose own token usage matched that conversation's context exactly — the request was ours, the text was not. Nothing on the platform recorded which upstream response it was, so neither we nor the provider could trace it afterwards. Bonds report this on done so the platform can log our request id next to the provider's own ids, and attach them to the message that gets persisted (or dropped).

Every field is optional: a provider that exposes none of them simply reports nothing, and the platform still has its own side of the correlation.

interface ProviderResponseMeta {
  /** The caller's `ChatParams.requestId`, echoed back (or the bond's own mint). */
  requestId?: string
  /** The provider's id for this completion, e.g. an OpenAI-compatible `chatcmpl`. */
  responseId?: string
  /** A transport-level trace id, when the provider returns one (e.g. `x-ds-trace-id`). */
  traceId?: string
  /**
   * The model id the PROVIDER says answered. A re-host may serve a different
   * snapshot than the id we asked for (dated vs. undated weights), so this is
   * recorded rather than assumed equal to the requested model.
   */
  model?: string
  /** The endpoint the request was sent to — which region/host actually served it. */
  endpoint?: string
}

TokenUsage

Token usage from a chat completion.

interface TokenUsage {
  inputTokens: number
  outputTokens: number
  /** Number of input tokens written to the prompt cache. */
  cacheCreationInputTokens?: number
  /** Number of input tokens read from the prompt cache. */
  cacheReadInputTokens?: number
  /**
   * Number of web searches the provider ran for this request through a server
   * tool (`ServerTool`, e.g. web search / search grounding). Providers bill
   * these per search, on top of tokens, so bonds MUST report every search the
   * provider executed — absent means none ran. Counted the way the provider
   * bills: Anthropic reports it (`server_tool_use.web_search_requests`),
   * OpenAI bills per `web_search_call`, Gemini per search query issued.
   */
  webSearchRequests?: number
  /**
   * The speed tier the provider REPORTS the request ran at (Anthropic wires it
   * as `usage.speed` per the fast-mode docs, and as `usage.service_tier` on
   * observed live responses — bonds read both). Only set by bonds whose
   * provider has a fast/priority tier — absent means unreported. Metering
   * prefers THIS field over the requested `ChatParams.speed`; when a
   * fast-requested turn succeeds with no report, callers may conservatively
   * bill the requested tier (Anthropic enforces fast-mode entitlement with a
   * hard 429, so a successful fast request was served fast).
   */
  speed?: 'standard' | 'fast'
}

AiRateLimitEvent

Details of one rate-limited or overloaded upstream response, reported via a provider config's onRateLimit callback each time the provider receives an HTTP 429/503 (or a provider-specific overload status such as Anthropic's 529), before any retry sleep. This surfaces the hits the provider's internal retry loop goes on to recover — the early-warning signal that capacity is running out, which a terminal error event alone would miss entirely.

interface AiRateLimitEvent {
  /** Provider name (e.g. 'anthropic'). */
  provider: string
  /** Model the rejected request targeted. */
  model: string
  /** HTTP status the upstream returned (429, 503, or 529). */
  status: number
  /** 1-based attempt number that was rejected. */
  attempt: number
  /** Whether the provider will retry this request again after a delay. */
  willRetry: boolean
  /** Delay before the next retry in milliseconds; 0 when `willRetry` is false. */
  retryInMs: number
  /** The response's parsed `Retry-After` header in seconds, when present and valid. */
  retryAfterSeconds?: number
}

AiStreamStallEvent

Details of one upstream stream that accepted the request and then produced no CONTENT for the whole first-token window — reported via a provider config's onStreamStall callback the moment the provider's first-token watchdog gives up on that attempt.

A stall is NOT an HTTP failure and NOT a rate limit: the provider answered 200 and opened a stream, then sat there. It is invisible to every other signal the platform has — a 200 with no content looks like a healthy request until the bytes stop — so without this callback a multi-minute pause cannot be told apart from a long think, a platform hold, or a retry backoff after the fact (X0 xr-molecule-flash-m71, 2026-10-16: a 3 min 15 s gap that production could not classify with any record it had — this callback is what makes the provider side of that question answerable, one way or the other).

interface AiStreamStallEvent {
  /** Provider name (e.g. 'deepseek'). */
  provider: string
  /** Model the stalled request targeted. */
  model: string
  /**
   * Which bound the stream broke: `before_content` is the first-token window
   * (retryable — the reply never started), `after_content` is the inter-chunk
   * silence window (not retryable — a partial reply is already on the wire).
   */
  phase: 'before_content' | 'after_content'
  /** How long the attempt waited for its first content event, in milliseconds. */
  stallMs: number
  /** 1-based attempt number that stalled. */
  attempt: number
  /** Whether the provider retries this request with a fresh stream. */
  willRetry: boolean
  /**
   * Content characters the stalled attempt streamed before it was abandoned —
   * always 0 for a first-token stall, and non-zero only for a post-content
   * stall, where a retry is impossible and the partial reply is kept.
   */
  streamedChars: number
  /**
   * Liveness frames the attempt received, counted as data arrived that carried
   * no content. Distinguishes the two stall shapes: 0 is a stream that went
   * completely silent, a high count is an upstream that is OPEN and dripping
   * empty frames (DeepSeek's `: keep-alive` comments during a long prefill),
   * which no byte-level idle timeout can see.
   */
  keepAlives: number
}

JSONSchema

JSON Schema subset for tool parameter definitions.

interface JSONSchema {
  type: string
  properties?: Record<string, JSONSchema>
  items?: JSONSchema
  required?: string[]
  description?: string
  enum?: unknown[]
  [key: string]: unknown
}

AITool

Tool definition that the AI model can invoke.

interface AITool {
  name: string
  description: string
  parameters: JSONSchema
  execute: (input: unknown) => Promise<unknown>
}

ServerTool

Provider-native tool handled server-side (e.g., Anthropic's web_search).

Unlike AITool, server tools are executed by the AI provider itself — no client-side execute callback is needed. The provider passes them through to the API alongside custom tools.

interface ServerTool {
  /**
   * Provider-specific tool type identifier — a VERSIONED string the provider
   * publishes (e.g. Anthropic's `"web_search_20260209"` on current models;
   * older models use earlier variants like `"web_search_20250305"`). Copy the
   * exact value from the provider's docs for the model in use.
   */
  type: string
  /** Tool name. */
  name: string
  /** Allow additional provider-specific fields (max_uses, etc.). */
  [key: string]: unknown
}

ChatParams

Parameters for a chat call.

interface ChatParams {
  messages: ChatMessage[]
  tools?: AITool[]
  /** Provider-native server tools (e.g. web search) — executed by the provider, not the caller. */
  serverTools?: ServerTool[]
  system?: string
  stream?: boolean
  maxTokens?: number
  temperature?: number
  model?: string
  /**
   * Enable extended thinking / reasoning on models that support it.
   *
   * `budgetTokens` is the abstract reasoning budget; bonds without a native
   * token-budget param translate it (e.g. via thresholds) into their provider's
   * control. `effort` — when present — is the PROVIDER-NATIVE effort value for
   * the active model, resolved by the caller from the model catalog (the model's
   * own `supportedEffortLevels`), e.g. Anthropic `output_config.effort`
   * (`'low' | 'medium' | 'high' | 'xhigh' | 'max'`) or an OpenAI-compatible
   * `reasoning_effort`. Bonds MUST prefer `effort` over
   * `budgetTokens` when set: on current Anthropic models (Fable 5 / Opus 4.8 /
   * Sonnet 5) a raw `budget_tokens` request is rejected with a 400 — adaptive
   * thinking + effort is the only control.
   */
  thinking?: { type: 'enabled'; budgetTokens: number; effort?: string }
  /**
   * Request the provider's fast/priority speed tier for this call (e.g.
   * Anthropic's `speed: "fast"` — same model, faster output, premium pricing).
   *
   * Callers should only set `'fast'` for models whose catalog entry declares
   * `fastPricing` (that field is the capability flag). Bonds whose provider has
   * no speed tier ignore it. Bonds that support it MUST report the speed the
   * provider says the request actually ran at in `TokenUsage.speed`, so
   * metering prices the served tier, not the requested one. No automatic
   * fallback is performed here: a fast-tier rate limit surfaces as a normal
   * error/retry, and the caller decides whether to drop back to standard.
   */
  speed?: 'standard' | 'fast'
  /** Enable prompt caching. Providers that support it will cache system prompts and tools. */
  cacheControl?: { type: 'ephemeral' }
  /** Abort signal to cancel in-flight API requests when the client disconnects. */
  signal?: AbortSignal
  /**
   * Control whether the model must call a tool. 'auto' (default) lets the model
   * decide; 'required' forces at least one tool call (any tool); `{ type: 'tool',
   * name }` forces that ONE specific tool — stronger than 'required', which only
   * guarantees *some* tool and can let the model drift to a different one when the
   * conversation history is biased toward it.
   */
  toolChoice?: 'auto' | 'required' | { type: 'tool'; name: string }
  /**
   * Sequences the provider stops generating at (`stop` on the OpenAI-compatible
   * wire). The provider drops the matched text from the reply — the sequence
   * itself is never returned — so this ends a reply the caller would discard
   * anyway, at the cost of the tokens written before the match instead of the
   * whole answer.
   *
   * Providers cap the list (DeepSeek and OpenAI accept 16 and 4 respectively);
   * a bond sends as many as its own API takes and ignores the rest. Bonds whose
   * provider has no stop parameter ignore it.
   */
  stop?: string[]
  /**
   * Opaque, stable identifier for the END USER on whose behalf this request is
   * made — forwarded to providers that accept one (Anthropic `metadata.user_id`,
   * OpenAI-compatible `user`).
   *
   * This is an ABUSE-ATTRIBUTION control, and it is the difference between a
   * provider suspending one account and suspending your organization's key. With
   * nothing sent, every request from every tenant is indistinguishable from the
   * platform itself, so the only enforcement action available to the provider is
   * against the key that serves all of them.
   *
   * It MUST be opaque — a hash/uuid, never an email, name, phone or anything else
   * that identifies a person to the provider — and it MUST be stable per user, so
   * a repeat offender is recognizable across sessions. Callers are expected to
   * derive it (e.g. an HMAC of their internal user id) and to keep the mapping on
   * their side so an abuse report naming this value can be traced back.
   *
   * Bonds whose provider has no equivalent field ignore it.
   */
  endUserId?: string
  /**
   * Correlation id for THIS call, minted by the caller and sent to the provider
   * so a reply can be tied back to the request that asked for it.
   *
   * The provider's own response id is not enough on its own: when a reply is
   * wrong (2026-10-16: a molecule.dev conversation received a complete, foreign
   * completion — a Flutter/.NET ticket review — from the provider), the vendor
   * can only trace it if we can name the request. Bonds that have a transport
   * header to carry it send it verbatim (they MUST NOT fold it into
   * `endUserId`, which is a stable per-user abuse-attribution value that every
   * request must share), capture whatever trace id the provider returns, and
   * report both back in the `done` event's `provider` field. Bonds with no such
   * channel still report the response's own id. Absent = the bond mints one.
   */
  requestId?: string
  /**
   * Extra provider-native request-body params, shallow-merged into the outgoing
   * request body as a BASE — the bond's own structural fields (model, messages,
   * tools, stream, and its computed token limit) are applied AFTER and always
   * win, so this can add or override tunables the SDK does not model
   * (`reasoning_effort`, `enable_thinking`, `top_k`, `top_p`, penalties, …)
   * without ever corrupting the wire protocol.
   *
   * Primarily for "bring your own AI" endpoints whose server accepts params
   * this SDK has no typed field for. Bonds merge it at the request-body ROOT;
   * for a nested-config wire format (Gemini) a `generationConfig` object here is
   * merged INTO the bond's `generationConfig` rather than replacing it. Values
   * must be JSON-serializable. Bonds that build a fixed body ignore it.
   */
  extraBody?: Record<string, unknown>
}

AIProvider

AI provider interface.

Each bond package (Anthropic, OpenAI, Gemini, etc.) implements this interface to provide model-specific chat functionality.

interface AIProvider {
  readonly name: string

  /**
   * Send a chat request and stream back events.
   *
   * Returns an async iterable of ChatEvent objects.
   * Always yields a final 'done' event with token usage.
   * @returns An async iterable that yields `ChatEvent` objects (text chunks, tool calls, done, or error).
   */
  chat(params: ChatParams): AsyncIterable<ChatEvent>
}

AIConfig

AI provider configuration.

interface AIConfig {
  /** Default model to use when not specified in ChatParams. */
  defaultModel?: string
  /** Maximum tokens for completions. */
  maxTokens?: number
  /** Default temperature. */
  temperature?: number
}

Types

ContentBlock

Rich content block within a message.

Includes text, tool interactions, and file attachments (images, documents, audio, video). Provider bonds map these generic blocks to their native API format (e.g., Anthropic base64 source, OpenAI image_url, etc.).

type ContentBlock =
  | { type: 'text'; text: string }
  | { type: 'image'; mediaType: string; data: string }
  | { type: 'document'; mediaType: string; data: string; filename?: string }
  | { type: 'audio'; mediaType: string; data: string }
  | { type: 'video'; mediaType: string; data: string }
  // `signature` is a provider-opaque replay token attached to the tool call
  // (Gemini 3.x `thoughtSignature`). Callers that persist tool calls and replay
  // them in later requests MUST carry it back on the replayed block verbatim —
  // Gemini rejects a replayed functionCall without it (400 "Function call is
  // missing a thought_signature"). Providers without such a token omit it, and
  // every bond ignores it when it has no native equivalent.
  | { type: 'tool_use'; id: string; name: string; input: unknown; signature?: string }
  | { type: 'tool_result'; tool_use_id: string; content: string | unknown }

ChatEvent

Streaming event from an AI chat call.

type ChatEvent =
  | { type: 'text'; content: string }
  | { type: 'thinking'; content: string }
  // `signature`: provider-opaque replay token for this tool call (see the
  // ContentBlock tool_use variant) — consumers must persist it alongside
  // id/name/input and echo it on the replayed tool_use block.
  | { type: 'tool_use'; id: string; name: string; input: unknown; signature?: string }
  // Emitted as soon as the model BEGINS a tool call — id + name are known but the
  // input is still streaming. Lets consumers show what's happening immediately
  // (e.g. "Writing the plan") instead of staring at a frozen spinner for the
  // seconds-to-minutes it takes to generate a large tool input (a file, a plan).
  | { type: 'tool_use_start'; id: string; name: string }
  // Progress for a tool call's input as the model streams its arguments:
  // `chars` is the number of input characters in this chunk. Carries real
  // progress (re-arms the stream-progress timeout; ticks the UI's live token
  // estimate) where previously these chunks produced only `keep_alive` (no
  // content) — the root cause of the dead loading indicator while a big tool
  // input was being written. `chars` is a COUNT, not the full content (the UI
  // only needs the magnitude, and forwarding the whole input would duplicate the
  // final `tool_use`). `text` is the raw partial-JSON CHUNK for this delta —
  // consumed SERVER-SIDE only (a coalescing consumer accumulates it to extract a
  // few short display fields, e.g. the file `path`, so the UI can label the
  // in-flight tool card before the args finish). It is never echoed wholesale to
  // the client. Optional so non-streaming/legacy providers can omit it.
  | { type: 'tool_input_delta'; id: string; chars: number; text?: string }
  // Incremental usage SNAPSHOT — the provider's own token counts as reported so
  // far on the wire (e.g. Anthropic's message_start carries the full input +
  // cache token counts before any output streams). LATEST WINS; `done.usage`
  // remains the authoritative final figure. METERING CONTRACT: consumers MUST
  // retain the latest snapshot and book it when a stream ends WITHOUT a `done`
  // (client abort, disconnect, progress timeout, provider error) — the upstream
  // provider bills those tokens even though the stream was cut, and dropping
  // them silently under-meters real spend. Providers whose wire protocol only
  // reports usage at stream end (OpenAI-compatible `include_usage`) cannot emit
  // mid-stream snapshots; consumers must estimate aborted turns for those.
  | { type: 'usage'; usage: TokenUsage }
  | { type: 'done'; usage: TokenUsage; provider?: ProviderResponseMeta }
  | { type: 'error'; message: string; errorKey?: string }
  // Liveness signal. PROVIDER CONTRACT: every streaming provider MUST yield
  // `keep_alive` whenever it receives data from the upstream API that produces
  // no other ChatEvent — e.g. an SSE ping/keepalive, an empty delta, or buffered
  // tool-input/argument chunks streaming in. Consumers use a (long) inter-event
  // timeout to detect a dead stream; without keep_alive that timeout false-fires
  // while the model is alive but producing only silent chunks (e.g. streaming a
  // large tool input). Not forwarded to end clients. Enforced by the provider
  // conformance test in this package's __tests__.
  | { type: 'keep_alive' }

AiRateLimitCallback

Callback invoked on each rate-limited/overloaded upstream response. Must not throw (providers guard it anyway); keep it fast — it runs on the request path.

type AiRateLimitCallback = (event: AiRateLimitEvent) => void

AiStreamStallCallback

Callback invoked on each stalled upstream stream. Must not throw (providers guard it anyway); keep it fast — it runs on the request path.

type AiStreamStallCallback = (event: AiStreamStallEvent) => void

Functions

setProvider(provider)

Registers an AI provider in singleton mode.

• Singleton: setProvider(provider) — bonds a single default provider.

function setProvider(provider: AIProvider): void

• provider — The default provider implementation for this process.

getProvider()

Retrieves the singleton AI provider, or null if none is bonded.

Falls back to a single named provider when no singleton is bonded — this lets apps that wire bond('ai', 'anthropic', provider) directly (without going through setProvider's singleton-fallback) still work with code that uses the simple getProvider() / requireProvider() accessors. When multiple named providers are bonded, the fallback declines (returns null) because the choice is ambiguous — those call sites must use getProviderByName(name) explicitly.

This also applies to an auto-promoted singleton: setProvider('a', p1) followed by setProvider('b', p2) does NOT leave getProvider() stuck returning p1 forever — once 'b' is registered the pick is genuinely ambiguous and getProvider() declines (null), same as if neither had been auto-promoted. An explicit setProvider(provider) singleton is unaffected by how many named providers exist.

function getProvider(): AIProvider | null

Returns: The bonded AI provider, or null.

getProviderByName(name)

Retrieves a named AI provider, or null if not bonded.

function getProviderByName(name: string): AIProvider | null

• name — The provider name (e.g. 'anthropic', 'xai').

Returns: The named AI provider, or null.

getAllProviders()

Retrieves all named AI providers as a Map keyed by provider name.

function getAllProviders(): Map<string, AIProvider>

Returns: Map of provider name → AIProvider.

hasProvider(name)

Checks whether an AI provider is currently bonded.

function hasProvider(name?: string): boolean

• name — Optional provider name. If omitted, checks the singleton.

Returns: true if the provider is bonded.

requireProvider()

Retrieves the bonded AI provider, throwing if none is bonded. Use this when AI functionality is required.

Routes through the same resolution as getProvider() so the single-named- bond fallback applies — apps that wire bond('ai', 'anthropic', provider) directly still satisfy this call without having to switch to the explicit getProviderByName() pattern. When resolution declined because MULTIPLE named providers are bonded with no explicit singleton, the thrown message says so (distinct from "nothing bonded at all") and points at getProviderByName().

function requireProvider(): AIProvider

Returns: The bonded AI provider.

Available Providers

Provider	Package
Alibaba Qwen	@molecule/api-ai-alibaba
Anthropic	@molecule/api-ai-anthropic
DeepSeek	@molecule/api-ai-deepseek
Google Gemini	@molecule/api-ai-google
Local (self-hosted)	@molecule/api-ai-local
MiniMax	@molecule/api-ai-minimax
Moonshot	@molecule/api-ai-moonshot
OpenAI	@molecule/api-ai-openai
xAI	@molecule/api-ai-xai
Zhipu GLM	@molecule/api-ai-zhipu

Injection Notes

Requirements

Peer dependencies:

• @molecule/api-bond ^1.0.1
• @molecule/api-i18n ^1.0.1

Runtime Dependencies

• @molecule/api-bond
• @molecule/api-i18n

The AI provider is a SERVER-side integration — a weak integration leaks the key, trusts the model, or gets billed:

• The provider API key is SERVER-ONLY. Call chat() from YOUR API and stream results to the browser (SSE); NEVER put the AI key in the frontend or call the provider directly from the browser — the key would ship to every user.
• Never blindly trust model output. Treat anything the model returns — code, SQL, a shell command, a URL, a tool-call argument — as UNTRUSTED input: validate/whitelist it, run it only in a sandbox, and re-check permissions server-side. User/content text in the prompt can hijack the model (prompt injection), so model output must never directly trigger a privileged action (delete, pay, email, exec) without your own authorization + validation.
• Gate + budget it. Require auth and rate-limit AI endpoints and cap maxTokens — an open, unauthenticated AI route is an unbounded bill.
• chat() returns an async iterable of ChatEvent (text chunks, tool calls, a final done with usage) — iterate it and forward chunks to the client.
• setProvider(name, provider) ambiguity: the FIRST named provider you register also auto-promotes to the singleton (so plain getProvider()/requireProvider() work without the caller knowing the name) — but that promotion is a single-provider convenience, not a permanent pick. The moment a SECOND, differently-named provider is registered, getProvider() stops returning the first one and declines (null) instead — requireProvider() throws pointing at getProviderByName(name). Call the explicit setProvider(provider) (no name) form if you want one provider to always win regardless of how many named providers you also register.

E2E Tests

Integration checklist — drive the real UI (live preview, no mocks), adapt each item to this app's actual chat/AI screens, and check every box off one by one. A box you can't check is an integration bug to fix — not a skip. The sandbox HAS an AI provider bonded, so the flow runs live end-to-end; AI output is NON-DETERMINISTIC, so assert on STRUCTURE/behavior, not exact text:

• A message sent through the real chat UI comes back as a RELEVANT AI reply — not an echo of the prompt, a hardcoded stub, or an empty bubble. Ask something with a checkable answer (e.g. "What is 2 + 2?") and confirm the response actually contains it ("4"), proving a live model answered.
• If the app streams, tokens render INCREMENTALLY — text grows word by word in the UI, not one final blob dumped after a long frozen spinner. (A streamed chat() yields text chunks then a final done; a single late blob means the reply was awaited whole and streaming is broken.)
• Multi-turn CONTEXT is preserved: a follow-up that refers back to the previous turn (e.g. after "2 + 2", ask "now double that" -> understood as 8) works — proving the full messages history is sent, not just the last line.
• A provider failure (bad/missing key, rate limit, timeout) surfaces as a graceful in-UI error message, NOT a crash, blank screen, a spinner that never resolves, or an unhandled 500. Force one and watch the UI recover.
• The provider key + the provider call are SERVER-side only: the key never reaches the browser (check the network tab, the JS bundle, and page globals), and no route proxies arbitrary prompts to the model without auth
  • a token cap — an open AI endpoint is an unbounded bill and abuse vector.

Translations

Translation strings are provided by @molecule/api-locales-ai.