@molecule/app-ai-voice-default

Default ai-voice provider for molecule.dev — browser Web Speech API for speech-to-text (SpeechRecognition) and text-to-speech (SpeechSynthesis).

Auto-generated, AI-first package reference for the molecule.dev ecosystem. It is written to be read by coding agents as much as by people, and is generated from this package's source — edit src/index.ts JSDoc, not this file.

Quick Start

import { setProvider } from '@molecule/app-ai-voice'
import { provider } from '@molecule/app-ai-voice-default'

setProvider(provider) // at startup — lazy; no config needed
// setProvider(createProvider({ ... })) to pass default recognition/synthesis options

Type

provider

Installation

npm install @molecule/app-ai-voice-default @molecule/app-ai-voice

API

Interfaces

`DefaultVoiceConfig`

Configuration specific to the default Web Speech API voice provider. Extends the base AIVoiceConfig with Web Speech API-specific options.

interface DefaultVoiceConfig extends AIVoiceConfig {
  /**
   * When true, immediately restarts recognition after it ends
   * (e.g. due to silence timeout) in continuous mode.
   * Defaults to true.
   */
  autoRestart?: boolean
}

Classes

`DefaultVoiceProvider`

Default voice provider implementation using the browser Web Speech API.

Provides speech-to-text via SpeechRecognition and text-to-speech via SpeechSynthesis. Falls back gracefully when APIs are unavailable.

class DefaultVoiceProvider implements AIVoiceProvider {
  readonly name = 'default'

  private state: VoiceState = 'idle'
  private config: DefaultVoiceConfig
  private recognition: SpeechRecognition | null = null
  private handlers: VoiceEventHandlers = {}
  private disposed = false
  private shouldRestart = false

  /**
   * Creates a new DefaultVoiceProvider.
   * @param config - Provider configuration with default recognition/synthesis options.
   */
  constructor(config: DefaultVoiceConfig = {}) {
    this.config = config
  }

  /**
   * Starts speech recognition using the Web Speech API.
   * @param options - Recognition options that override defaults from config.
   * @param handlers - Callbacks for transcript results, state changes, and errors.
   */
  startListening(options?: VoiceRecognitionOptions, handlers?: VoiceEventHandlers): void {
    if (this.disposed) return

    const Ctor = getSpeechRecognitionCtor()
    if (!Ctor) {
      handlers?.onError?.({
        code: 'not-supported',
        message: 'Speech recognition is not supported in this browser',
      })
      return
    }

    // Stop any existing recognition session before starting a new one
    this.stopListening()

    this.handlers = handlers ?? {}
    const merged = { ...this.config.recognition, ...options }

    const recognition = new Ctor()
    recognition.lang = merged.language ?? 'en-US'
    recognition.continuous = merged.continuous ?? false
    recognition.interimResults = merged.interimResults ?? true
    recognition.maxAlternatives = merged.maxAlternatives ?? 1

    this.shouldRestart = this.config.autoRestart !== false && (merged.continuous ?? false)

    recognition.onresult = (event: SpeechRecognitionEvent) => {
      const last = event.results[event.results.length - 1]
      if (!last) return
      const alt = last[0]
      if (!alt) return
      this.handlers.onTranscript?.({
        transcript: alt.transcript,
        isFinal: last.isFinal,
        confidence: alt.confidence,
      })
    }

    recognition.onerror = (event: SpeechRecognitionErrorEvent) => {
      // 'aborted' fires when we call stop() intentionally — not a real error
      if (event.error === 'aborted') return

      // 'no-speech' in continuous mode is recoverable — just keep listening
      if (event.error === 'no-speech' && this.shouldRestart) return

      this.setState('error')
      this.handlers.onError?.({
        code: event.error,
        message: event.message || `Speech recognition error: ${event.error}`,
      })
    }

    recognition.onstart = () => {
      this.setState('listening')
    }

    recognition.onend = () => {
      // Auto-restart in continuous mode if we haven't been explicitly stopped
      if (this.shouldRestart && !this.disposed && this.state === 'listening') {
        try {
          recognition.start()
        } catch (_error) {
          // Auto-restart failed (e.g. recognition already running); fall back to idle safely
          this.setState('idle')
        }
        return
      }
      if (this.state !== 'error') {
        this.setState('idle')
      }
    }

    this.recognition = recognition

    try {
      recognition.start()
    } catch (err) {
      this.handlers.onError?.({
        code: 'start-failed',
        message: err instanceof Error ? err.message : 'Failed to start speech recognition',
      })
    }
  }

  /**
   * Stops the current speech recognition session.
   */
  stopListening(): void {
    this.shouldRestart = false
    if (this.recognition) {
      try {
        this.recognition.abort()
      } catch (_error) {
        // Already stopped — abort() on a non-running recognition is safe to ignore
      }
      this.recognition = null
    }
    if (this.state === 'listening') {
      this.setState('idle')
    }
  }

  /**
   * Speaks the given text using the Web Speech Synthesis API.
   * @param text - The text to speak aloud.
   * @param options - Synthesis options that override defaults from config.
   * @returns A promise that resolves when speech finishes or is interrupted.
   */
  async speak(text: string, options?: VoiceSynthesisOptions): Promise<void> {
    if (this.disposed) return

    const synth = getSpeechSynthesis()
    if (!synth) {
      throw new Error('Speech synthesis is not supported in this browser')
    }

    // Cancel any current speech
    synth.cancel()

    const merged = { ...this.config.synthesis, ...options }
    const Utterance = getUtteranceCtor()
    if (!Utterance) {
      throw new Error('Speech synthesis is not supported in this browser')
    }
    const utterance = new Utterance(text)
    utterance.lang = merged.language ?? 'en-US'
    utterance.rate = merged.rate ?? 1
    utterance.pitch = merged.pitch ?? 1
    utterance.volume = merged.volume ?? 1

    // Resolve the voice by name if specified
    if (merged.voice) {
      const voices = synth.getVoices()
      const match = voices.find((v) => v.name === merged.voice || v.voiceURI === merged.voice)
      if (match) {
        utterance.voice = match
      }
    }

    this.setState('speaking')

    return new Promise<void>((resolve, reject) => {
      utterance.onend = () => {
        if (this.state === 'speaking') {
          this.setState('idle')
        }
        this.handlers.onSpeakEnd?.()
        resolve()
      }

      utterance.onerror = (event) => {
        // 'canceled' fires when we call cancel() intentionally
        if (event.error === 'canceled') {
          if (this.state === 'speaking') {
            this.setState('idle')
          }
          resolve()
          return
        }
        this.setState('error')
        this.handlers.onError?.({
          code: event.error,
          message: `Speech synthesis error: ${event.error}`,
        })
        reject(new Error(`Speech synthesis error: ${event.error}`))
      }

      synth.speak(utterance)
    })
  }

  /**
   * Stops any current speech synthesis.
   */
  stopSpeaking(): void {
    const synth = getSpeechSynthesis()
    if (synth) {
      synth.cancel()
    }
    if (this.state === 'speaking') {
      this.setState('idle')
    }
  }

  /**
   * Returns the current voice provider state.
   * @returns The current VoiceState.
   */
  getState(): VoiceState {
    return this.state
  }

  /**
   * Checks whether any voice feature (recognition or synthesis) is supported.
   * @returns True if at least one API is available.
   */
  isSupported(): boolean {
    return this.isRecognitionSupported() || this.isSynthesisSupported()
  }

  /**
   * Checks whether the SpeechRecognition API is available.
   * @returns True if speech recognition is supported.
   */
  isRecognitionSupported(): boolean {
    return getSpeechRecognitionCtor() !== null
  }

  /**
   * Checks whether the SpeechSynthesis API is available.
   * @returns True if speech synthesis is supported.
   */
  isSynthesisSupported(): boolean {
    return getSpeechSynthesis() !== null
  }

  /**
   * Returns the list of available speech synthesis voices.
   * Waits for voices to load if they haven't yet (some browsers load them asynchronously).
   * @returns A promise resolving to an array of VoiceDescriptor objects.
   */
  async getAvailableVoices(): Promise<VoiceDescriptor[]> {
    const synth = getSpeechSynthesis()
    if (!synth) return []

    let voices = synth.getVoices()
    if (voices.length === 0) {
      // Some browsers (Chrome) load voices asynchronously
      voices = await new Promise<SpeechSynthesisVoice[]>((resolve) => {
        const onVoicesChanged = (): void => {
          synth.removeEventListener('voiceschanged', onVoicesChanged)
          resolve(synth.getVoices())
        }
        synth.addEventListener('voiceschanged', onVoicesChanged)

        // Timeout after 2 seconds if voiceschanged never fires
        setTimeout(() => {
          synth.removeEventListener('voiceschanged', onVoicesChanged)
          resolve(synth.getVoices())
        }, 2000)
      })
    }

    return voices.map((v) => ({
// … truncated — the full declaration is in node_modules/@molecule/app-ai-voice-default/dist/provider.d.ts

Functions

`createProvider(config)`

Creates a DefaultVoiceProvider instance.

function createProvider(config?: DefaultVoiceConfig): DefaultVoiceProvider

config — Optional configuration with default recognition/synthesis options.

Returns: A DefaultVoiceProvider that uses the browser Web Speech API.

Constants

`provider`

The provider implementation — the fleet-standard typed provider const.

Wire it once at startup: setProvider(provider) from @molecule/app-ai-voice. It is a lazy proxy: construction is deferred to the first property access, so importing this module never throws and needs no config up front. Use createProvider(config) instead when you need default recognition/synthesis options.

const provider: AIVoiceProvider

Core Interface

Implements @molecule/app-ai-voice interface.

Bond Wiring

Setup function to register this provider with the core interface:

import { setProvider } from '@molecule/app-ai-voice'
import { provider } from '@molecule/app-ai-voice-default'

export function setupAiVoiceDefault(): void {
  setProvider(provider)
}

Injection Notes

Requirements

Peer dependencies:

@molecule/app-ai-voice ^1.0.1

Runtime Dependencies

@molecule/app-ai-voice

Failure modes are asymmetric: when recognition is unsupported, startListening() reports { code: 'not-supported' } through handlers.onError and returns — but when synthesis is unsupported, speak() THROWS. Feature-detect with isRecognitionSupported() / isSynthesisSupported() and wrap speak() in try/catch. speak() cancels any utterance already playing. autoRestart (default true) only applies with continuous: true, where transient 'no-speech' ends are swallowed and listening resumes automatically — call stopListening() to actually stop. options.voice matches by voice name or voiceURI from getAvailableVoices().

E2E Tests

Integration checklist — drive the real UI (live preview, no mocks), adapt each item to this app's actual screens/flows, and check every box off one by one. A box you can't check is an integration bug to fix — not a skip:

Pressing the mic / press-to-talk control calls startListening() and speech appears as a live transcript in the UI — interim VoiceTranscriptEvent updates (isFinal: false) refresh the text as you speak, and the final one (isFinal: true) commits the recognized text via onTranscript.
Stopping (stopListening()) halts recognition cleanly: the transcript stops updating, the mic control returns to idle, and no stray final result fires afterward.
Denying mic permission (or unavailable hardware) fires onError with a VoiceErrorEvent (code: 'not-allowed') and shows a visible message — the mic control never sits as a silent dead button.
The app's text-to-speech action calls speak(text, ...) and you actually hear the given text; the chosen VoiceDescriptor / VoiceSynthesisOptions are honored (voice, language, and rate change the audible output), and stopSpeaking() cuts it off.
The recognition VoiceRecognitionOptions language is respected — setting it to a non-default locale (e.g. 'fr-FR') recognizes in that language rather than always defaulting to English.
A visible listening/speaking indicator tracks getState() / onStateChange — it reads 'listening' while the mic is open and 'speaking' during synthesis, and returns to 'idle' when each ends.
Voice UI is feature-gated on isRecognitionSupported() / isSynthesisSupported() (and getAvailableVoices() is awaited, not read synchronously) so an unsupported browser hides the control instead of throwing.
Microphone access is requested only from a user gesture, its denial is handled gracefully, and captured audio/transcripts stay within the session — nothing is logged or sent anywhere the app didn't intend.

`@molecule/app-ai-voice-default`

How it works

Reference