Molecule.dev

@molecule/api-model-hosting-modal

Provider bond · model-hosting · API (Node) · v1.0.1 · Apache-2.0

Modal model hosting for molecule.dev — run a model server container on a T4/L4/A10 GPU or CPU with per-second billing, behind Modal proxy auth

npm install @molecule/api-model-hosting-modal

npm · Source on GitHub · Implements @molecule/api-model-hosting

How it works

@molecule/api-model-hosting-modal is a provider bond on the API (Node) side: it implements the model-hosting core interface (@molecule/api-model-hosting) with a concrete vendor or library behind it.

Your code calls the core; you wire this provider once at startup. Swapping vendors later is one line in that wiring, not a rewrite.

import { setProvider, requireProvider } from '@molecule/api-model-hosting'
import { provider as modal } from '@molecule/api-model-hosting-modal'

setProvider('modal', modal) // reads MODAL_TOKEN_ID/SECRET + MODAL_WORKSPACE on first use

const endpoint = await requireProvider().deploy({
  name: 'kev',
  image: 'ghcr.io/acme/kev-serve:0.8.2',
  // Modal starts the server from Python, so the command is required:
  command: ['kev-serve', '--port', '8000'],
  port: 8000,
  healthPath: '/health',
  accelerator: { kind: 'gpu', minVramGb: 16 }, // smallest fit → T4
  region: 'us',
  scaling: { minInstances: 0, maxInstances: 2, idleTimeoutSeconds: 300 },
  access: 'private',
})
const headers = await requireProvider().authHeaders(endpoint.id) // { 'Modal-Key': …, 'Modal-Secret': … }

Works with: @molecule/api-bond, @molecule/api-i18n, @molecule/api-model-hosting, @molecule/api-secrets

Secrets: MODAL_TOKEN_ID, MODAL_TOKEN_SECRET, MODAL_WORKSPACE, MODAL_PROXY_TOKEN_ID (optional), MODAL_PROXY_TOKEN_SECRET (optional)

Reference

Modal model hosting for molecule.dev — run a model server container on a T4/L4/A10… GPU or CPU with per-second billing, scale to zero, behind Modal proxy auth.

Modal is the cheap place for small GPU models (Kev, Jeff, Intern-Decision — a T4 fits a 0.8–4B fine-tune) and the easiest place to keep one warm-cheap: containers boot in about a second, idle windows go down to 2 s, and $30/mo of compute is free on the Starter plan.

Quick Start

import { setProvider, requireProvider } from '@molecule/api-model-hosting'
import { provider as modal } from '@molecule/api-model-hosting-modal'

setProvider('modal', modal) // reads MODAL_TOKEN_ID/SECRET + MODAL_WORKSPACE on first use

const endpoint = await requireProvider().deploy({
  name: 'kev',
  image: 'ghcr.io/acme/kev-serve:0.8.2',
  // Modal starts the server from Python, so the command is required:
  command: ['kev-serve', '--port', '8000'],
  port: 8000,
  healthPath: '/health',
  accelerator: { kind: 'gpu', minVramGb: 16 }, // smallest fit → T4
  region: 'us',
  scaling: { minInstances: 0, maxInstances: 2, idleTimeoutSeconds: 300 },
  access: 'private',
})
const headers = await requireProvider().authHeaders(endpoint.id) // { 'Modal-Key': …, 'Modal-Secret': … }

Type

provider

Installation

npm install @molecule/api-model-hosting-modal @molecule/api-bond @molecule/api-i18n @molecule/api-model-hosting @molecule/api-secrets

API

Interfaces

ModalAppRow

One app as modal app list --json reports it (only the fields we read).

interface ModalAppRow {
  /** The app's `ap-…` id. */
  app_id?: string
  /** The app name (what `modal.App(name)` set). */
  description?: string
  /** `deployed`, `stopped`, `ephemeral`, `disabled`, … */
  state?: string
  /** ISO timestamp, when reported. */
  created_at?: string
}

ModalConfig

Configuration for the Modal model hosting provider.

interface ModalConfig {
  /** Modal token id. Defaults to `MODAL_TOKEN_ID`. */
  tokenId?: string
  /** Modal token secret. Defaults to `MODAL_TOKEN_SECRET`. */
  tokenSecret?: string
  /** Workspace name, for endpoint URLs. Defaults to `MODAL_WORKSPACE`. */
  workspace?: string
  /** Modal environment to deploy to. Defaults to `MODAL_ENVIRONMENT`, then the workspace default. */
  environment?: string
  /**
   * The environment's web suffix, when it has one (`<workspace>-<suffix>--<label>.modal.run`).
   * Defaults to `MODAL_ENVIRONMENT_SUFFIX`; empty for the default environment.
   */
  environmentSuffix?: string
  /** Proxy auth token id, for `access: 'private'` calls. Defaults to `MODAL_PROXY_TOKEN_ID`. */
  proxyTokenId?: string
  /** Proxy auth token secret. Defaults to `MODAL_PROXY_TOKEN_SECRET`. */
  proxyTokenSecret?: string
  /** `modal` CLI binary. Default `'modal'`. */
  modalBin?: string
  /** Python binary with the `modal` package installed (for `scale`). Default `'python3'`. */
  pythonBin?: string
  /**
   * Add a Python interpreter to a registry image that has none
   * (`Image.from_registry(…, add_python=…)`), e.g. `'3.12'`. Default: not added.
   */
  addPython?: string
  /** Seconds the server may take to start listening before Modal gives up. Default 600. */
  startupTimeoutSeconds?: number
  /** Custom runner (tests). */
  runner?: ModalRunner
  /** How often `deploy` probes the health path, in ms. Default 5000. */
  pollIntervalMs?: number
}

RunResult

Result of one CLI / Python run.

interface RunResult {
  /** Exit code (`null` when killed). */
  code: number | null
  /** Standard output. */
  stdout: string
  /** Standard error. */
  stderr: string
}

Types

ModalRunner

Runs one command (the modal CLI or python3) with extra environment. Swappable for tests, or to run Modal from a sidecar.

type ModalRunner = (
  bin: string,
  args: string[],
  opts: { env: Record<string, string>; cwd?: string; input?: string },
) => Promise<RunResult>

Classes

ModalClient

Talks to Modal by running its CLI / Python bridge.

Functions

appSource(spec, opts)

The app.py for a spec.

function appSource(
  spec: ModelEndpointSpec,
  opts: { addPython?: string; startupTimeoutSeconds: number },
): string
  • spec — The endpoint spec (command required: Modal starts the server from Python).
  • opts — Python and startup options.
  • opts.addPython — Python version to add to an image without one.
  • opts.startupTimeoutSeconds — How long the server may take to listen.

Returns: Python source.

createProvider(config)

Creates a Modal model hosting provider.

function createProvider(config?: ModalConfig): ModelHostingProvider
  • config — Modal configuration; env vars fill anything omitted.

Returns: A ModelHostingProvider backed by Modal.

endpointUrl(cfg, label)

The web URL for a label: https://<workspace>[-<suffix>]--<label>.modal.run (modal.com/docs/guide/webhook-urls, 2026-09-30 — the source part carries the environment suffix when the environment has one).

function endpointUrl(cfg: { workspace?: string; environmentSuffix?: string }, label: string): string
  • cfg — Resolved config.
  • label — The label= the app deployed with.

Returns: The endpoint base URL.

fromAppRow(row, cfg)

Maps a modal app list row to an endpoint.

function fromAppRow(
  row: { description?: string; state?: string; app_id?: string; created_at?: string },
  cfg: { workspace?: string; environmentSuffix?: string },
): ModelEndpoint | null
  • row — The row.
  • cfg — Workspace/suffix for the URL.

Returns: The endpoint, or null for an app that is not serving.

gpuFor(spec)

The Modal GPU string for a spec: the named one, or the smallest that fits.

function gpuFor(spec: ModelEndpointSpec): string | null
  • spec — The endpoint spec.

Returns: A GPU string, or null for CPU.

regionMultiplier(region)

The region multiplier Modal charges: 1.15 for the us / eu aliases, the quoted 1.75 upper bound for any narrower named region.

function regionMultiplier(region: string): number

resolveModalConfig(config, env)

Resolves config with env fallbacks.

function resolveModalConfig(
  config: ModalConfig,
  env?: NodeJS.ProcessEnv,
): Required<
  Pick<ModalConfig, 'modalBin' | 'pythonBin' | 'pollIntervalMs' | 'startupTimeoutSeconds'>
> & {
  tokenId?: string
  tokenSecret?: string
  workspace?: string
  environment?: string
  environmentSuffix?: string
  proxyTokenId?: string
  proxyTokenSecret?: string
  addPython?: string
  runner?: ModalRunner
}
  • config — Explicit config.
  • env — The process environment (swappable for tests).

Returns: Every setting filled.

toPython(value)

Renders a JSON-shaped value as a Python literal.

function toPython(value: unknown): string
  • value — The value.

Returns: Python source.

Constants

What Modal offers here (modal.com/docs/guide/gpu + /guide/cold-start, 2026-09-30).

const MODAL_CAPABILITIES: ProviderCapabilities

The generated app's single web function name.

const MODAL_FUNCTION_NAME: 'serve'

GPU types by memory, smallest first.

const MODAL_GPUS: readonly { id: string; vramGb: number }[]

Dated Modal list prices, USD per second (modal.com/pricing, 2026-09-30; quotes: "Nvidia T4 $0.000164 / sec", "Nvidia L4 $0.000222 / sec", "CPU Physical core (2 vCPU equivalent) $0.0000131 / core / sec", "$0.00000222 / GiB / hour" memory, "Region selection 1.15 - 1.75x base prices"). GPUs without a quoted price here are refused by estimate rather than guessed.

const MODAL_PRICES: {
  gpusPerSecond: Record<string, number>
  cpuPerCoreSecond: number
  memoryPerGiBSecond: number
}

modelHostingModalSecretDefinitions

Secret definitions used by the Modal model hosting bond.

const modelHostingModalSecretDefinitions: SecretDefinition[]

provider

The provider implementation — lazy, so env vars are read on first use.

const provider: ModelHostingProvider

Core Interface

Implements @molecule/api-model-hosting interface.

Bond Wiring

Setup function to register this provider with the core interface:

import { setProvider } from '@molecule/api-model-hosting'
import { provider } from '@molecule/api-model-hosting-modal'

export function setupModelHostingModal(): void {
  setProvider(provider)
}

Injection Notes

Requirements

Peer dependencies:

  • @molecule/api-bond ^1.0.1
  • @molecule/api-i18n ^1.0.1
  • @molecule/api-model-hosting ^1.0.0
  • @molecule/api-secrets ^1.0.1

Environment Variables

  • MODAL_TOKEN_ID (required) — modal token id
    • Setup: Modal token id — authenticates the modal CLI this bond deploys with (modal.com/settings → API tokens → "New token" prints both parts).
    • Get it here: https://modal.com/settings
    • Example: ak-XXXXXXXXXXXXXXXXXXXXXXXX
  • MODAL_TOKEN_SECRET (required) — modal token secret
    • Setup: Modal token secret — the other half of the Modal CLI token.
    • Get it here: https://modal.com/settings
    • Example: as-XXXXXXXXXXXXXXXXXXXXXXXX
  • MODAL_WORKSPACE (required) — modal workspace
    • Setup: Modal workspace name — the first part of every endpoint URL (https://<workspace>--<label>.modal.run). Shown top-left of the Modal dashboard.
    • Get it here: https://modal.com/settings
    • Example: my-workspace
  • MODAL_PROXY_TOKEN_ID (optional) — modal proxy token id
  • MODAL_PROXY_TOKEN_SECRET (optional) — modal proxy token secret

Runtime Dependencies

  • @molecule/api-bond

  • @molecule/api-i18n

  • @molecule/api-model-hosting

  • @molecule/api-secrets

  • Deploys run the modal CLI — Modal has no REST deploy API. The machine calling deploy needs the modal binary (and, for scale, a python3 with the modal package). The CLI authenticates with MODAL_TOKEN_ID + MODAL_TOKEN_SECRET; the endpoint URL needs MODAL_WORKSPACE (and MODAL_ENVIRONMENT / MODAL_ENVIRONMENT_SUFFIX when not deploying to the default environment).

  • command is required — Modal starts your server from a Python function (@modal.web_server wraps the image), so it cannot just run the image's entrypoint. An image without a Python interpreter needs createProvider({ addPython: '3.12' }) (Image.from_registry(…, add_python=…)).

  • scale rewrites the live autoscaler (Function.update_autoscaler) — min/max/idle take effect without a redeploy. Per-instance concurrency is a decorator and needs a fresh deploy. scale on an endpoint this process did not deploy cannot find the app to scale — it is the one method that is not purely id-driven.

  • There is no delete on Modal — remove stops the app (serves nothing, bills nothing); the name stays taken.

  • authHeaders returns Modal proxy-auth headers (Modal-Key / Modal-Secret from MODAL_PROXY_TOKEN_ID / MODAL_PROXY_TOKEN_SECRET) for access: 'private' endpoints — Modal calls this "proxy auth" (requires_proxy_auth=True). Feed them to the ai-decisions bonds' headers hook per request. access: 'public' needs no headers, and is reachable by anyone with the URL.

  • list() is environment-wide: Modal cannot mark which apps this bond created, so it lists every deployed app in the configured environment. Region/hardware/scaling read back correctly only for endpoints this process deployed.

  • Billing is per container-second, not per request: with minInstances ≥ 1 you pay around the clock; with minInstances: 0 only while containers are up (requests + the scaledown tail). estimate prices the first exactly and the second by busy time alone.

  • GPUs price at T4 $0.000164/s and L4 $0.000222/s (verified 2026-09-30); other GPU ids deploy fine but estimate refuses them rather than guess. Region pinning costs 1.15× (us/eu) to 1.75×.