@molecule/api-model-hosting-runpod

RunPod Serverless model hosting for molecule.dev — run a model server container on a load-balancing GPU endpoint (16 GB class and up) that scales to zero, called at https://<endpoint-id>.api.runpod.ai with your RunPod API key.

Load-balancing endpoints run YOUR HTTP server (no RunPod handler code), so laya-serve, jeff-serve or any /v1/systemone server works unchanged.

Quick Start

import { setProvider, requireProvider } from '@molecule/api-model-hosting'
import { provider } from '@molecule/api-model-hosting-runpod'

setProvider(provider) // reads RUNPOD_API_KEY on first use

const endpoint = await requireProvider().deploy({
  name: 'jeff',
  image: 'ghcr.io/acme/jeff-serve:0.8b',
  port: 8000,
  healthPath: '/ping',
  accelerator: { kind: 'gpu', minVramGb: 16 },
  region: 'us',
  scaling: { minInstances: 0, maxInstances: 2, idleTimeoutSeconds: 60 },
  access: 'private',
})
const headers = await requireProvider().authHeaders(endpoint.id) // { authorization: 'Bearer <RUNPOD_API_KEY>' }

Type

provider

Installation

npm install @molecule/api-model-hosting-runpod @molecule/api-bond @molecule/api-i18n @molecule/api-model-hosting @molecule/api-secrets

API

Interfaces

`RunPodConfig`

Configuration for the RunPod model hosting provider.

interface RunPodConfig {
  /** API key. Defaults to `RUNPOD_API_KEY`. */
  apiKey?: string
  /** GraphQL `locations` for `'us'`. Default `'US'`. */
  usLocations?: string
  /** GraphQL `locations` for `'eu'`. Default `'CZ,FR,GB,NO,RO'`. */
  euLocations?: string
  /** Container disk per worker, in GB. Default 20. */
  containerDiskGb?: number
  /** REST base URL (tests). Default `https://rest.runpod.io/v1`. */
  restBaseUrl?: string
  /** GraphQL URL (tests). Default `https://api.runpod.io/graphql`. */
  graphqlUrl?: string
  /** Endpoint URL template (tests); `{id}` is replaced. Default `https://{id}.api.runpod.ai`. */
  endpointUrlTemplate?: string
  /** How often `deploy` probes the health path while workers start, in ms. Default 5000. */
  pollIntervalMs?: number
}

`RunPodEndpoint`

A RunPod serverless endpoint (only the fields read here).

interface RunPodEndpoint {
  id: string
  name: string
  templateId?: string
  workersMin?: number
  workersMax?: number
  idleTimeout?: number
  gpuTypeIds?: string[]
  dataCenterIds?: string[]
  createdAt?: string
  workers?: unknown[]
}

`RunPodTemplate`

A RunPod template (only the fields read here).

interface RunPodTemplate {
  id: string
  name: string
}

Classes

`RunPodApiError`

A RunPod API error, carrying the HTTP status.

`RunPodClient`

Thin RunPod client.

Functions

`createProvider(config)`

Creates a RunPod model hosting provider.

function createProvider(config?: RunPodConfig): ModelHostingProvider

config — API key, locations, disk size.

Returns: A ModelHostingProvider backed by RunPod Serverless.

`gpuIdsFor(spec)`

The GPU pools a spec may land on: the named pool, or every pool with enough memory.

function gpuIdsFor(spec: ModelEndpointSpec): string

spec — The endpoint spec.

Returns: Comma-separated gpuIds.

`toGraphQLLiteral(value)`

Serializes a plain object as a GraphQL input literal (unquoted keys, type values that are enums stay quoted strings — RunPod takes strings).

function toGraphQLLiteral(value: unknown): string

value — The value.

Returns: The GraphQL literal.

`toTemplate(spec, diskGb)`

The serverless template body for a spec.

function toTemplate(spec: ModelEndpointSpec, diskGb: number): Record<string, unknown>

spec — The endpoint spec.
diskGb — Container disk.

Returns: TemplateCreateInput.

Constants

`modelHostingRunPodSecretDefinitions`

Secret definitions used by the RunPod model hosting bond.

const modelHostingRunPodSecretDefinitions: SecretDefinition[]

`provider`

The provider implementation — lazy, so env vars are read on first use.

const provider: ModelHostingProvider

`RUNPOD_CAPABILITIES`

What RunPod Serverless load-balancing endpoints offer here (GPU only).

const RUNPOD_CAPABILITIES: ProviderCapabilities

`RUNPOD_GPU_POOLS`

GPU pools by memory, smallest first (docs.runpod.io/sdks/graphql/manage-endpoints, 2026-09-30).

const RUNPOD_GPU_POOLS: readonly { id: string; vramGb: number }[]

`RUNPOD_GRAPHQL`

Default GraphQL URL.

const RUNPOD_GRAPHQL: 'https://api.runpod.io/graphql'

`RUNPOD_REST`

Default REST base URL.

const RUNPOD_REST: 'https://rest.runpod.io/v1'

Core Interface

Implements @molecule/api-model-hosting interface.

Bond Wiring

Setup function to register this provider with the core interface:

import { setProvider } from '@molecule/api-model-hosting'
import { provider } from '@molecule/api-model-hosting-runpod'

export function setupModelHostingRunpod(): void {
  setProvider(provider)
}

Injection Notes

Requirements

Peer dependencies:

@molecule/api-bond ^1.0.1
@molecule/api-i18n ^1.0.1
@molecule/api-model-hosting ^1.0.0
@molecule/api-secrets ^1.0.1

Environment Variables

RUNPOD_API_KEY (required) — runpod api key
- Setup: RunPod API key — Manages templates and serverless endpoints, and authorizes calls to them (Settings → API Keys). Workers stop when the account balance reaches $0.
- Get it here: https://www.console.runpod.io/user/settings
- Example: rpa_XXXXXXXXXXXXXXXXXXXXXXXX

Runtime Dependencies

@molecule/api-bond
@molecule/api-i18n
@molecule/api-model-hosting
@molecule/api-secrets
Serve /ping on your port. RunPod's load balancer health-polls /ping (the HEALTH_CHECK_PATH override is documented but reported ignored — runpod/docs#853) and expects 200 healthy / 204 still loading. A server with only /health never becomes healthy; add /ping (or proxy it). The bond sets PORT, PORT_HEALTH and HEALTH_CHECK_PATH from the spec anyway, and deploy waits on YOUR healthPath through the public URL.
Cold starts return "no workers available" until a worker is up; retry 3 times 5–10 s apart (RunPod's own guidance), or keep minInstances: 1.
GPU only. minVramGb picks the smallest pool class that fits (AMPERE_16 → 16 GB; 24 GB → AMPERE_24,ADA_24; …); name one or a comma-separated set in accelerator.model. 'us' → US, 'eu' → CZ,FR,GB,NO,RO, or pass RunPod location codes.
The endpoint is created with the GraphQL saveEndpoint mutation (type: "LB"), because the REST API cannot create load-balancing endpoints; updates, reads and deletes use REST. remove drains workers to 0 before deleting, then deletes the template.
Anyone with your RunPod API key can call every endpoint — the key is both the admin credential and the endpoint credential. Keep it server-side.
Workers stop when the account balance reaches $0 — a silent outage.
list() returns every serverless endpoint on the account (RunPod does not say which are load-balancing).
API errors throw RunPodApiError with the HTTP status.

`@molecule/api-model-hosting-runpod`

How it works

Reference