@molecule/api-model-hosting-cloud-run

Google Cloud Run model hosting for molecule.dev — run a model server container on CPU or an NVIDIA L4 GPU, scaled between min and max instances (including zero), behind Google IAM.

Cloud Run is the cheap place for small encoder models (Laya, GLiNER) on CPU: request-based billing charges only while a request is being served, with a monthly free tier. The same API runs an L4 GPU (instance-billed) when a Qwen-based model needs CUDA.

Quick Start

import { setProvider, requireProvider } from '@molecule/api-model-hosting'
import { provider } from '@molecule/api-model-hosting-cloud-run'

setProvider(provider) // reads GOOGLE_CLOUD_PROJECT + GOOGLE_SERVICE_ACCOUNT_JSON on first use

const endpoint = await requireProvider().deploy({
  name: 'laya',
  image: 'us-docker.pkg.dev/acme/models/laya-serve:0.3.1',
  port: 8000,
  healthPath: '/health',
  accelerator: { kind: 'cpu' },
  region: 'us',
  scaling: { minInstances: 0, maxInstances: 3, idleTimeoutSeconds: 900 },
  access: 'private',
})
const headers = await requireProvider().authHeaders(endpoint.id) // { authorization: 'Bearer <ID token>' }

Type

provider

Installation

npm install @molecule/api-model-hosting-cloud-run @molecule/api-bond @molecule/api-i18n @molecule/api-model-hosting @molecule/api-secrets

API

Interfaces

CloudRunConfig

Configuration for the Cloud Run model hosting provider.

interface CloudRunConfig {
  /** Google Cloud project id. Defaults to `GOOGLE_CLOUD_PROJECT`. */
  projectId?: string
  /**
   * Service account key JSON (the file's contents, not a path). Defaults to
   * `GOOGLE_SERVICE_ACCOUNT_JSON`. Needs `roles/run.admin` to deploy and
   * `roles/run.invoker` on the services to call private ones.
   */
  serviceAccountJson?: string
  /** Region for `'us'`. Default `'us-central1'` (has L4 GPUs). */
  usRegion?: string
  /** Region for `'eu'`. Default `'europe-west4'` (has L4 GPUs). */
  euRegion?: string
  /** Service account email the containers run as (optional; Cloud Run's default otherwise). */
  runtimeServiceAccount?: string
  /** Admin API base URL (tests, proxies). Default `https://run.googleapis.com`. */
  apiBaseUrl?: string
  /** OAuth token endpoint (tests). Default `https://oauth2.googleapis.com/token`. */
  tokenUrl?: string
  /** How often `deploy` polls the service while it rolls out, in ms. Default 5000. */
  pollIntervalMs?: number
}

RunCondition

A Cloud Run v2 Condition (only the fields read here).

interface RunCondition {
  type?: string
  state?: 'CONDITION_PENDING' | 'CONDITION_RECONCILING' | 'CONDITION_FAILED' | 'CONDITION_SUCCEEDED'
  message?: string
}

RunService

A Cloud Run v2 Service (only the fields read or written here).

interface RunService {
  name?: string
  uri?: string
  createTime?: string
  reconciling?: boolean
  terminalCondition?: RunCondition
  labels?: Record<string, string>
  annotations?: Record<string, string>
  ingress?: string
  invokerIamDisabled?: boolean
  template?: {
    containers?: Array<Record<string, unknown>>
    scaling?: { minInstanceCount?: number; maxInstanceCount?: number }
    nodeSelector?: { accelerator?: string }
    gpuZonalRedundancyDisabled?: boolean
    maxInstanceRequestConcurrency?: number
    serviceAccount?: string
  }
}

ServiceAccountKey

The fields of a service account key this bond reads.

interface ServiceAccountKey {
  client_email: string
  private_key: string
  private_key_id?: string
}

Classes

CloudRunApiError

A Cloud Run API error, carrying the vendor's HTTP status and status code.

CloudRunClient

Thin Admin API v2 client.

GoogleTokenSource

Caches Google tokens until a minute before they expire.

Functions

createProvider(config)

Creates a Cloud Run model hosting provider.

function createProvider(config?: CloudRunConfig): ModelHostingProvider

• config — Project, service account key, region mapping.

Returns: A ModelHostingProvider backed by Cloud Run.

fromRunService(service)

Maps a Cloud Run service to a model endpoint.

function fromRunService(service: RunService): ModelEndpoint

• service — The service as the API returns it.

Returns: The endpoint.

parseServiceAccountKey(raw)

Parses the service account key JSON, refusing anything without the two fields used.

function parseServiceAccountKey(raw: string | undefined): ServiceAccountKey

• raw — The key file contents.

Returns: The parsed key.

signServiceAccountJwt(key, claims, tokenUrl, now)

Signs a service-account JWT (RS256).

function signServiceAccountJwt(
  key: ServiceAccountKey,
  claims: Record<string, string>,
  tokenUrl: string,
  now?: number,
): string

• key — The service account key.
• claims — Extra claims (scope or target_audience).
• tokenUrl — The aud claim.
• now — Current time in seconds (tests).

Returns: The compact JWT.

toRunService(spec, runtimeServiceAccount)

Builds the Cloud Run v2 Service body for a spec.

function toRunService(spec: ModelEndpointSpec, runtimeServiceAccount?: string): RunService

• spec — The endpoint spec.
• runtimeServiceAccount — Optional service account the container runs as.

Returns: The service body.

Constants

CLOUD_PLATFORM_SCOPE

Scope for the Admin API.

const CLOUD_PLATFORM_SCOPE: 'https://www.googleapis.com/auth/cloud-platform'

CLOUD_RUN_API

Default Admin API base URL.

const CLOUD_RUN_API: 'https://run.googleapis.com'

CLOUD_RUN_CAPABILITIES

What Cloud Run offers here.

const CLOUD_RUN_CAPABILITIES: ProviderCapabilities

CLOUD_RUN_GPU_REGIONS

Regions with L4 GPUs (docs.cloud.google.com/run/docs/configuring/services/gpu, 2026-09-30).

const CLOUD_RUN_GPU_REGIONS: readonly [
  'asia-southeast1',
  'asia-south1',
  'europe-west1',
  'europe-west4',
  'us-central1',
  'us-east4',
]

GOOGLE_TOKEN_URL

Default OAuth token endpoint.

const GOOGLE_TOKEN_URL: 'https://oauth2.googleapis.com/token'

MANAGED_LABEL

Label every service this bond creates carries, so list returns only those.

const MANAGED_LABEL: 'molecule-model-hosting'

modelHostingCloudRunSecretDefinitions

Secret definitions used by the Cloud Run model hosting bond.

const modelHostingCloudRunSecretDefinitions: SecretDefinition[]

provider

The provider implementation — lazy, so env vars are read on first use.

const provider: ModelHostingProvider

Core Interface

Implements @molecule/api-model-hosting interface.

Bond Wiring

Setup function to register this provider with the core interface:

import { setProvider } from '@molecule/api-model-hosting'
import { provider } from '@molecule/api-model-hosting-cloud-run'

export function setupModelHostingCloudRun(): void {
  setProvider(provider)
}

Injection Notes

Requirements

Peer dependencies:

• @molecule/api-bond ^1.0.1
• @molecule/api-i18n ^1.0.1
• @molecule/api-model-hosting ^1.0.0
• @molecule/api-secrets ^1.0.1

Environment Variables

• GOOGLE_CLOUD_PROJECT (required) — google cloud project
  • Setup: Google Cloud project — The project id the model endpoints run in (Cloud Run API enabled, billing on).
  • Get it here: https://console.cloud.google.com/projectselector2/home/dashboard
  • Example: acme-models-123456
• GOOGLE_SERVICE_ACCOUNT_JSON (required) — google service account json
  • Setup: Google service account key — The key file contents (JSON) of a service account with the Cloud Run Admin role, and Cloud Run Invoker on the services it calls.
  • Get it here: https://console.cloud.google.com/iam-admin/serviceaccounts
  • Example: {"type":"service_account","client_email":"…","project_id":"acme-models"}

Runtime Dependencies

• @molecule/api-bond

• @molecule/api-i18n

• @molecule/api-model-hosting

• @molecule/api-secrets

• Credentials: GOOGLE_CLOUD_PROJECT and GOOGLE_SERVICE_ACCOUNT_JSON (the key file CONTENTS). The account needs Cloud Run Admin to deploy and Cloud Run Invoker on the services to call private ones; the Cloud Run API must be enabled and billing on.

• Images must be pullable by Cloud Run — Artifact Registry in the same project, or a public registry. A private GHCR/Docker Hub image fails the deploy with a pull error in the endpoint's error.

• authHeaders mints a Google-signed ID token for the service URL and caches it for about an hour. Call it per request (the ai-decisions bonds' headers hook) — never store the value. access: 'public' sets invokerIamDisabled and authHeaders returns {}.

• GPU = NVIDIA L4 only here, in us-central1, us-east4, europe-west1, europe-west4, asia-south1, asia-southeast1 ('us' → us-central1, 'eu' → europe-west4). It forces ≥ 4 vCPU / 16 GiB and instance-based billing, and zonal redundancy is turned off (cheaper; a zone outage takes it down). New projects start with a small GPU quota.

• idleTimeoutSeconds is ignored — Cloud Run decides when idle instances stop. minInstances: 0 means no instance (and, on CPU, no cost) while idle.

• secretEnv becomes plain environment variables on the revision, readable by anyone with viewer access to the project. For real secrets use Secret Manager references yourself.

• deploy waits for the Ready condition (startup probe on healthPath, up to 10 minutes of probing) and polls every 5 s; a failed rollout returns status: 'failed' with Cloud Run's message.

• list() returns only services this bond created (label managed-by=molecule-model-hosting) in the us/eu regions configured.

• Errors from the API throw CloudRunApiError with the HTTP status and Google's error code (e.g. PERMISSION_DENIED).