@molecule/api-ai-speech-molecule

molecule.dev hosted speech provider for @molecule/api-ai-speech.

Text-to-speech and transcription run on molecule.dev and are billed to your molecule project, so the app needs no OpenAI account. It is an ordinary bond: swap it for @molecule/api-ai-speech-openai (your own key) or @molecule/api-ai-speech-elevenlabs without changing code that calls the core.

Auto-generated, AI-first package reference for the molecule.dev ecosystem. It is written to be read by coding agents as much as by people, and is generated from this package's source — edit src/index.ts JSDoc, not this file.

Quick Start

import { setProvider, requireProvider } from '@molecule/api-ai-speech'
import { provider } from '@molecule/api-ai-speech-molecule'

setProvider(provider) // reads MOLECULE_API_KEY from the environment

const speech = requireProvider()
const { audio, contentType } = await speech.synthesize!({
  input: 'Your order shipped.',
  voice: 'nova',
})
const { text, duration } = await speech.transcribe!({ audio: uploadedBytes, filename: 'memo.m4a' })

Type

provider

Installation

npm install @molecule/api-ai-speech-molecule @molecule/api-ai-speech @molecule/api-secrets

API

Interfaces

MoleculeSpeechConfig

Options for createProvider. Every field falls back to an env var, so the zero-argument provider export works from .env alone.

interface MoleculeSpeechConfig {
  /**
   * The molecule project API key (`mk_…`), or the sandbox token molecule.dev
   * writes for in-IDE apps (`mbk_…`). Defaults to `MOLECULE_API_KEY`.
   */
  apiKey?: string
  /**
   * The hosted services base URL, without a trailing slash. Defaults to
   * `MOLECULE_SERVICES_URL`, then `https://api.molecule.dev/api/v1/services`.
   */
  servicesUrl?: string
  /** Default text-to-speech voice. Default `alloy`. */
  defaultVoice?: string
  /** Default text-to-speech model: `tts-1` (default) or `tts-1-hd`. */
  defaultTTSModel?: string
  /** Per-request timeout in milliseconds. Default 120000. */
  timeoutMs?: number
}

ProcessEnv

Environment variables this provider reads.

interface ProcessEnv {
  MOLECULE_API_KEY?: string
  MOLECULE_SERVICES_URL?: string
}

Classes

MoleculeServiceError

Error thrown for a refused or failed hosted-service call.

class MoleculeServiceError extends Error {
  /** HTTP status from molecule.dev. */
  readonly status: number
  /** Stable error key from molecule.dev, e.g. `broker.error.budgetExceeded`. */
  readonly errorKey: string | undefined

  /**
   * Create the error.
   *
   * @param message - Human-readable message.
   * @param status - HTTP status.
   * @param errorKey - Stable error key, when the service sent one.
   * @param cause - The underlying error, if any.
   */
  constructor(message: string, status: number, errorKey?: string, cause?: unknown) {
    super(message, cause === undefined ? undefined : { cause })
    this.name = 'MoleculeServiceError'
    this.status = status
    this.errorKey = errorKey
  }
}

MoleculeSpeechProvider

Speech through molecule.dev's hosted service.

class MoleculeSpeechProvider implements AISpeechProvider {
  readonly name = 'molecule'

  private readonly apiKey: string
  private readonly servicesUrl: string
  private readonly defaultVoice: string
  private readonly defaultTTSModel: string
  private readonly timeoutMs: number

  /**
   * Create the provider.
   *
   * @param config - Options; each falls back to its env var.
   */
  constructor(config: MoleculeSpeechConfig = {}) {
    this.apiKey = config.apiKey ?? process.env.MOLECULE_API_KEY ?? ''
    this.servicesUrl = resolveServicesUrl(
      config.servicesUrl ?? process.env.MOLECULE_SERVICES_URL ?? DEFAULT_SERVICES_URL,
    )
    this.defaultVoice = config.defaultVoice ?? 'alloy'
    this.defaultTTSModel = config.defaultTTSModel ?? 'tts-1'
    this.timeoutMs = config.timeoutMs ?? 120_000
  }

  /**
   * Text to speech. Voices: alloy, ash, coral, echo, fable, nova, onyx, sage,
   * shimmer. Models: tts-1, tts-1-hd. `instructions` is not supported by these
   * models and is not sent.
   *
   * @param params - Text, voice, model, format and speed.
   * @returns The audio bytes and their content type.
   */
  async synthesize(params: SynthesizeParams): Promise<SynthesizeResult> {
    const data = (await this.request('synthesize', {
      input: params.input,
      voice: params.voice ?? this.defaultVoice,
      model: params.model ?? this.defaultTTSModel,
      ...(params.responseFormat ? { responseFormat: params.responseFormat } : {}),
      ...(params.speed ? { speed: params.speed } : {}),
    })) as { audio: string; contentType: string }
    return {
      audio: new Uint8Array(Buffer.from(data.audio, 'base64')),
      contentType: data.contentType,
    }
  }

  /**
   * Speech to text with whisper-1. Always returns the verbose shape (text,
   * language, duration, segments); `responseFormat` and `timestampGranularity`
   * are ignored.
   *
   * @param params - Audio bytes, filename (for its extension), language, prompt.
   * @returns The transcription.
   */
  async transcribe(params: TranscribeParams): Promise<TranscribeResult> {
    if (params.audio.byteLength > SPEECH_SERVICE_LIMITS.maxTranscribeBytes) {
      throw new MoleculeServiceError(
        `The audio is ${params.audio.byteLength} bytes; the hosted service accepts at most ${SPEECH_SERVICE_LIMITS.maxTranscribeBytes}. ${HINTS[413]}`,
        413,
        'hostedServices.error.inputTooLarge',
      )
    }
    return (await this.request('transcribe', {
      audio: Buffer.from(params.audio).toString('base64'),
      ...(params.filename ? { filename: params.filename } : {}),
      ...(params.model ? { model: params.model } : {}),
      ...(params.language ? { language: params.language } : {}),
      ...(params.prompt ? { prompt: params.prompt } : {}),
    })) as TranscribeResult
  }

  /** One POST to the hosted service. */
  private async request(operation: string, body: Record<string, unknown>): Promise<unknown> {
    if (!this.apiKey) {
      throw new MoleculeServiceError(
        'MOLECULE_API_KEY is not set. Create a project API key on molecule.dev (or with `mlcl apikey create`) and put it in the API .env.',
        401,
        'hostedServices.error.tokenInvalid',
      )
    }
    const response = await fetch(`${this.servicesUrl}/speech/${operation}`, {
      method: 'POST',
      headers: { authorization: `Bearer ${this.apiKey}`, 'content-type': 'application/json' },
      body: JSON.stringify(body),
      signal: AbortSignal.timeout(this.timeoutMs),
    })
    const text = await response.text()
    let data: unknown
    try {
      data = text ? JSON.parse(text) : {}
    } catch (error) {
      // A proxy/gateway error page, not the service: report the status only.
      throw new MoleculeServiceError(
        `molecule.dev returned ${response.status} with a non-JSON body.`,
        response.status,
        undefined,
        error,
      )
    }
    if (!response.ok) {
      const err = data as { error?: string; errorKey?: string }
      const hint = HINTS[response.status]
      throw new MoleculeServiceError(
        `molecule.dev speech/${operation} failed (${response.status}): ${err.error ?? 'error'}${hint ? ` ${hint}` : ''}`,
        response.status,
        err.errorKey,
      )
    }
    return data
  }
}

Functions

createProvider(config)

Create a hosted speech provider.

function createProvider(config?: MoleculeSpeechConfig): AISpeechProvider

• config — Options; each falls back to its env var.

Returns: An AISpeechProvider backed by molecule.dev.

Constants

provider

The provider implementation.

const provider: AISpeechProvider

DEFAULT_SERVICES_URL

Default hosted services base URL.

const DEFAULT_SERVICES_URL: 'https://api.molecule.dev/api/v1/services'

SPEECH_SERVICE_LIMITS

Per-request limits of the hosted service.

const SPEECH_SERVICE_LIMITS: {
  readonly maxSynthesizeChars: 4096
  readonly maxTranscribeBytes: number
}

aiSpeechMoleculeSecretDefinitions

Secret definitions required by the molecule.dev hosted speech bond.

const aiSpeechMoleculeSecretDefinitions: SecretDefinition[]

Core Interface

Implements @molecule/api-ai-speech interface.

Bond Wiring

Setup function to register this provider with the core interface:

import { setProvider } from '@molecule/api-ai-speech'
import { provider } from '@molecule/api-ai-speech-molecule'

export function setupAiSpeechMolecule(): void {
  setProvider(provider)
}

Injection Notes

Requirements

Peer dependencies:

• @molecule/api-ai-speech >=1.0.1
• @molecule/api-secrets ^1.0.1

Environment Variables

• MOLECULE_API_KEY (required) — molecule.dev project API key
  • Setup: Create one for your project with mlcl apikey create --project <id> --name <name> (keys start with mk_).
  • Get it here: https://www.molecule.dev
  • Example: mk_...

Runtime Dependencies

• @molecule/api-ai-speech

• @molecule/api-secrets

• Config: MOLECULE_API_KEY (SERVER-side only) — a molecule project API key (mk_…) with scope broker or broker:speech. Optional MOLECULE_SERVICES_URL (default https://api.molecule.dev/api/v1/services; required — plain-http is refused unless the host is loopback or a private-network endpoint (RFC 1918 / *.docker.internal, e.g. the sandbox gateway host.docker.internal)).

• Implements ONLY synthesize and transcribe. synthesizeSpeech, synthesizeStream, listVoices and translate are absent — check before calling (if (speech.listVoices)), or bond the ElevenLabs provider for them.

• synthesize: models tts-1 (default) and tts-1-hd; voices alloy, ash, coral, echo, fable, nova, onyx, sage, shimmer; formats mp3 (default), opus, aac, flac, wav, pcm; at most 4096 characters per call — split long text at sentence boundaries. instructions (a gpt-4o-mini-tts feature) is not sent.

• transcribe: whisper-1 only; always returns text, language, duration and segments. Pass filename with the right extension (.m4a, .webm, …) — the service uses it to recognise the format. At most 12 MB per call.

• Errors are MoleculeServiceError with status and errorKey. 401: bad, revoked, or wrong-scope key. 402: the project owner's included allowance is used up — usage billing must be enabled on molecule.dev. 413: input too large. 429 / 503: slow down / retry later. Nothing is retried for you.

• Never import this from browser code: the key would ship to every visitor. Browser recordings go to YOUR API route, which calls transcribe.

E2E Tests

Integration checklist — drive the real UI (live preview, no mocks), adapt each item to this app's actual screens/flows, and check every box off one by one. A box you can't check is an integration bug to fix — not a skip. The wired provider implements a subset (TTS, STT, or both) — feature-detect and run only the direction(s) it actually exposes:

• TTS: each read-aloud / narration / voice-note flow the app defines returns real PLAYABLE audio — the response carries an audio Content-Type (e.g. audio/mpeg) and non-trivial bytes (not a 0-byte file, not a JSON-encoded blob), and the UI's <audio>/player actually plays it. The spoken audio reflects the input text you sent.
• TTS: synthesizing two different strings yields two audibly different clips (distinct byte payloads) — proves the text drives synthesis and you are not hearing a cached/placeholder file.
• TTS: any exposed voice/format/speed option takes effect — switching voice (voice/voiceId) or output format (responseFormat/ outputFormat) changes the returned audio (audible voice, Content-Type, or file extension); an unknown voice/format surfaces a visible error rather than silently falling back.
• STT: transcribing a KNOWN spoken clip returns text that MATCHES the words spoken (not empty, not garbage) and renders in the UI; translating a non-English clip returns English text. Any timestamps the UI shows line up with the audio.
• STT: a silent / empty / too-short clip is handled gracefully — an empty or "no speech detected" result shown in the UI, never a crash or a spinner that hangs forever.
• The audio artifact is served from the app's OWN origin (a stored upload or a route the app serves), NOT a raw expiring provider URL — copy the audio src, reload it later, and confirm it still plays.
• A provider error (bad key, quota, unsupported format/voice, or a capability the wired provider lacks) surfaces as a readable message in the UI, not an unhandled crash or a blank/broken player.
• Synthesis/transcription runs SERVER-SIDE and authorized: the provider key never reaches the browser (check the network tab / client bundle), and no unauthenticated or unbounded endpoint lets a caller fire arbitrary synthesize/transcribe calls — each is billed per character / per minute of audio, so an open endpoint is a cost-DoS vector.