@molecule/api-code-sandbox-flyio
Provider bond · code-sandbox · API (Node) · v1.3.0 · Apache-2.0
Fly.io Machines (Firecracker microVM) code sandbox provider
npm install @molecule/api-code-sandbox-flyionpm · Source on GitHub · Implements @molecule/api-code-sandbox
How it works
@molecule/api-code-sandbox-flyio is a provider bond on the API (Node) side: it implements the code-sandbox core interface (@molecule/api-code-sandbox) with a concrete vendor or library behind it.
Your code calls the core; you wire this provider once at startup. Swapping vendors later is one line in that wiring, not a rewrite.
import { bond } from '@molecule/api-bond'
import { createProvider } from '@molecule/api-code-sandbox-flyio'
// The API token is read from FLY_API_TOKEN (or FLY_ACCESS_TOKEN) unless you
// pass `apiToken` explicitly. Every option below has an env fallback too.
bond(
'code-sandbox',
createProvider({
orgSlug: 'my-org',
region: 'iad',
baseImage: 'registry.fly.io/molecule-sandbox:latest',
}),
)
// Elsewhere, through the core interface:
import { requireProvider } from '@molecule/api-code-sandbox'
const sandbox = await requireProvider().create({
projectId: 'a3f1c0de-0000-4000-8000-000000000001',
volumeName: 'mol-a3f1c0de',
resources: { cpu: 2, memoryMB: 2048, diskMB: 10240 },
})
await sandbox.exec('npm install', { timeout: 600_000 })
await sandbox.sleep() // Fly suspend — memory snapshot, storage-only billing
await sandbox.wake() // Fly start — resumes from the snapshot
// Warm start. Capture the prepared filesystem once…
const provider = requireProvider()
await provider.commitTemplate?.({
sandboxId: sandbox.id,
templateId: 'react-postgres-v3',
// REQUIRED here: the archive of these paths IS the template.
capturePaths: ['/workspace'],
})
// …then every later boot of the same configuration restores it instead of
// re-running `mlcl create` + `npm install`. A missing template THROWS.
const warm = await provider.create({
projectId: 'b7d2…',
volumeName: 'mol-b7d2',
templateId: 'react-postgres-v3',
})Works with: @molecule/api-bond, @molecule/api-i18n
Secrets: FLY_API_TOKEN, FLY_ORG_SLUG, FLY_REGION (optional), FLY_SANDBOX_IMAGE (optional), FLY_API_HOSTNAME (optional), FLY_SANDBOX_EGRESS_ALLOWED_PORTS (optional), FLY_SANDBOX_PRIVATE_SERVICES (optional)
Reference
Fly.io Machines code-sandbox provider for molecule.dev.
Runs each sandbox as a Fly Machine — a Firecracker microVM — managed over the
Fly Machines API. The reason to pick this over the Docker bond is
SCALE-TO-ZERO: sleep() maps to Fly's suspend, which snapshots the microVM's
memory to disk, and a suspended Machine bills for storage only. Waking is a
resume from that snapshot rather than a cold boot.
Auto-generated, AI-first package reference for the molecule.dev ecosystem. It is written to be read by coding agents as much as by people, and is generated from this package's source — edit
src/index.tsJSDoc, not this file.
Quick Start
import { bond } from '@molecule/api-bond'
import { createProvider } from '@molecule/api-code-sandbox-flyio'
// The API token is read from FLY_API_TOKEN (or FLY_ACCESS_TOKEN) unless you
// pass `apiToken` explicitly. Every option below has an env fallback too.
bond(
'code-sandbox',
createProvider({
orgSlug: 'my-org',
region: 'iad',
baseImage: 'registry.fly.io/molecule-sandbox:latest',
}),
)
// Elsewhere, through the core interface:
import { requireProvider } from '@molecule/api-code-sandbox'
const sandbox = await requireProvider().create({
projectId: 'a3f1c0de-0000-4000-8000-000000000001',
volumeName: 'mol-a3f1c0de',
resources: { cpu: 2, memoryMB: 2048, diskMB: 10240 },
})
await sandbox.exec('npm install', { timeout: 600_000 })
await sandbox.sleep() // Fly suspend — memory snapshot, storage-only billing
await sandbox.wake() // Fly start — resumes from the snapshot
// Warm start. Capture the prepared filesystem once…
const provider = requireProvider()
await provider.commitTemplate?.({
sandboxId: sandbox.id,
templateId: 'react-postgres-v3',
// REQUIRED here: the archive of these paths IS the template.
capturePaths: ['/workspace'],
})
// …then every later boot of the same configuration restores it instead of
// re-running `mlcl create` + `npm install`. A missing template THROWS.
const warm = await provider.create({
projectId: 'b7d2…',
volumeName: 'mol-b7d2',
templateId: 'react-postgres-v3',
})
Type
provider
Installation
npm install @molecule/api-code-sandbox-flyio @aws-sdk/client-s3 @aws-sdk/s3-request-presigner @molecule/api-bond @molecule/api-code-sandbox @molecule/api-i18n @molecule/api-proxy-agent
API
Interfaces
FlyRequestOptions
Options for a single Fly API request.
interface FlyRequestOptions {
/** HTTP method. Defaults to `GET`. */
method?: string
/** JSON request body. Omitted entirely when undefined. */
body?: unknown
/** Per-request timeout in ms. Defaults to the client's configured timeout. */
timeoutMs?: number
/** Max attempts for a retryable failure. Defaults to 4. Pass 1 to disable retries. */
attempts?: number
/**
* Treat these HTTP statuses as a successful `null` result instead of throwing.
* Used for idempotent deletes and existence checks (`[404]`).
*/
nullOn?: number[]
/**
* Extra HTTP statuses to treat as transient (retry) beyond the default
* `429`/`5xx`. The exec endpoint passes `[404]`: a machine-not-found for a
* Machine we just created and are actively driving is Fly API inconsistency
* under exec-burst load, not a real absence, so it is worth repeating.
*/
retryStatuses?: number[]
}
FlyApiClientOptions
Constructor options for FlyApiClient.
interface FlyApiClientOptions {
/** Bearer token. Resolved lazily so env/secrets can land after construction. */
token: () => string | undefined
/** Base URL including `/v1`. */
baseUrl: string
/** Default per-request timeout (ms). */
timeoutMs?: number
/** Injectable fetch, for tests. Defaults to the global `fetch`. */
fetchImpl?: typeof fetch
/** Injectable sleep, for tests. Defaults to `setTimeout`. */
sleep?: (ms: number) => Promise<void>
}
EgressProbeTarget
One literal host:port the probe attempts a raw TCP connection to.
interface EgressProbeTarget {
/** A literal IPv4 or IPv6 address. Never a hostname — see `parseEgressProbeTargets`. */
host: string
/** TCP port. */
port: number
}
EgressProbeContext
Context verdictForProbeExit renders its operator-facing messages from.
interface EgressProbeContext {
/** Fly app the probe Machine ran in. */
app: string
/** Targets the probe attempted. */
targets: EgressProbeTarget[]
/**
* Ports this provider's own egress policy allows, or `undefined` when it
* applies no policy at all. Used for the message, the remediation, and as the
* INTENT half of the drift check against `appliedPolicyPorts` — never to
* decide `open` vs `filtered` from a probe that reached a target, which comes
* from the observation alone.
*/
policyPorts?: FlyNetworkPolicyPort[]
/**
* Ports the policy Fly ACTUALLY holds for this app allows, read back from the
* API after the probe ran, or `undefined` when the readback failed or found no
* policy of this provider's name.
*
* This is the observation half of the only drift this module can catch: a
* policy widened outside the provider (by hand, by another tool, by an older
* build) allows ports nothing here configured, and the raw-connect probe
* cannot see it because it only attempts the targets it was given. See
* `unexpectedPolicyPorts`.
*/
appliedPolicyPorts?: FlyNetworkPolicyPort[]
}
StoredObject
One object as this bond needs to see it.
interface StoredObject {
/** Full key, including the configured prefix. */
key: string
/** Size in bytes. */
size: number
/** Last-modified timestamp, ISO 8601, or `null` when the store did not report one. */
lastModified: string | null
}
ObjectStore
The object-store operations the template capability needs.
Declared as an interface so templates.ts stays transport-free and its tests
can drive a fake store, the same way exec.ts takes an injected RawExec.
interface ObjectStore {
/** Human-readable identification of the store, for error messages. */
readonly describe: string
/** Bucket holding the templates. Used to render the opaque `SandboxTemplate.ref`. */
readonly bucket: string
/** Key prefix every template object lives under, with no trailing slash. */
readonly prefix: string
/**
* Every object under a prefix, following pagination to the end.
* THROWS on failure — a caller that deletes must never read a failed query as
* "nothing is there".
*/
list(prefix: string): Promise<StoredObject[]>
/** One object's metadata, or `null` when it does not exist. Throws on any other failure. */
head(key: string): Promise<StoredObject | null>
/** An object's body as text, or `null` when it does not exist. Throws on any other failure. */
getText(key: string): Promise<string | null>
/** Write a small text object. */
putText(key: string, body: string, contentType: string): Promise<void>
/** Delete objects. Deleting a key that is not there is a success. */
remove(keys: string[]): Promise<void>
/** A presigned URL a sandbox can `PUT` an archive to. */
presignPut(key: string, expiresInSeconds: number): Promise<string>
/** A presigned URL a sandbox can `GET` an archive from. */
presignGet(key: string, expiresInSeconds: number): Promise<string>
}
TemplateManifest
The control-plane-written record of a template. Its existence defines the template's existence.
interface TemplateManifest {
/** Schema version of this record. */
schema: number
/** The caller's template id. */
id: string
/**
* Absolute paths captured, exactly as the caller named them.
*
* This is the security-relevant field: on restore these become `tar`'s member
* selectors, so the extraction surface is control-plane policy rather than
* anything the captured sandbox could influence.
*/
capturePaths: string[]
/** When the capture completed, ISO 8601. */
createdAt: string
/** Archive size in bytes as observed in the store at capture time. */
sizeBytes: number
/** Free-form label recorded at capture time. */
label?: string
}
TemplateContext
What the template capability needs from the provider.
interface TemplateContext {
/** The configured object store, or `null` when the operator has not configured one. */
store: ObjectStore | null
/** Runs a shell command inside a Machine and returns its result. */
exec(app: string, machineId: string, command: string, timeoutMs: number): Promise<ExecResult>
/** Blocks until a Machine is running, so it can be exec'd into. */
ensureStarted(app: string, machineId: string): Promise<void>
/** Splits a composite sandbox id into its app and Machine id. */
parseSandboxId(id: string): { app: string; machineId: string }
/** Lifetime of a presigned capture/restore URL, in seconds. */
presignExpirySeconds: number
/** Wall-clock budget for a capture or restore transfer, in ms. */
transferTimeoutMs: number
/** Largest archive this provider will store, in bytes. */
maxArchiveBytes: number
warn?: (message: string, meta?: Record<string, unknown>) => void
debug?: (message: string, meta?: Record<string, unknown>) => void
}
FlyioConfig
Configuration for the Fly.io Machines sandbox provider.
Every option is also readable from an environment variable (see
ProcessEnv); explicit config always wins.
interface FlyioConfig {
/**
* Fly API token (sent as `Authorization: Bearer <token>`). Falls back to
* `FLY_API_TOKEN`, then `FLY_ACCESS_TOKEN`. Required — the provider throws a
* named error on the first API call if none is resolvable.
*/
apiToken?: string
/**
* Machines API base URL, INCLUDING the `/v1` path segment. Defaults to
* `FLY_API_HOSTNAME` (with `/v1` appended when the value has no path) or
* `https://api.machines.dev/v1`. From inside a Fly private network the
* documented internal endpoint is `http://_api.internal:4280/v1`.
*/
apiUrl?: string
/**
* Fly organization slug that owns the sandbox apps. Falls back to
* `FLY_ORG_SLUG`, then `personal`. Required for app creation and for
* `FlyioConfig.appPerProject` listing.
*/
orgSlug?: string
/**
* Shared Fly app that holds every sandbox Machine. Only used when
* `appPerProject` is `false`. Falls back to `FLY_SANDBOX_APP`.
*
* A shared app puts every tenant on ONE 6PN private network, where each
* sandbox can reach every other sandbox's dev-server ports by private IPv6.
* That is the same cross-tenant exposure the Docker bond's `bridge` network
* has, so this mode is REFUSED in production.
*/
appName?: string
/**
* Name prefix for the per-project Fly app when `appPerProject` is `true`
* (default). The app is `<appPrefix>-<sanitized projectId>`. Falls back to
* `FLY_SANDBOX_APP_PREFIX`, then `mol-sandbox`.
*/
appPrefix?: string
/**
* Create one Fly app — on its own custom 6PN private network — per project,
* so tenants are network-isolated from each other. Defaults to `true`.
* Set `false` (or `FLY_SANDBOX_APP_PER_PROJECT=false`) only for a
* single-tenant deployment; it is refused in production.
*/
appPerProject?: boolean
/**
* Custom 6PN network name for a per-project app. Defaults to the app name, so
* every project lands on its own private network. Ignored when
* `appPerProject` is `false`. A Fly app's network CANNOT be changed after
* creation.
*/
network?: string
/** Fly region for Machines and volumes (e.g. `iad`). Falls back to `FLY_REGION`, then `iad`. */
region?: string
/**
* OCI image every sandbox Machine runs. Must be pullable by Fly — a tag in
* the org's `registry.fly.io` repository or a public registry. A LOCAL
* `molecule-sandbox:latest` is not reachable by Fly; push it first. Falls
* back to `FLY_SANDBOX_IMAGE`, then `registry.fly.io/molecule-sandbox:latest`.
*/
baseImage?: string
/** Default vCPU count (`guest.cpus`). Defaults to 1. */
defaultCpu?: number
/** Default `guest.cpu_kind` — `shared` or `performance`. Defaults to `shared`. */
defaultCpuKind?: string
/** Default memory in MB (`guest.memory_mb`). Defaults to 1024. */
defaultMemoryMB?: number
/**
* Size in GB of the volume created for a sandbox's `/workspace` when
* `SandboxConfig.volumeName` is set. Fly volumes are sized in whole GB, so the
* core's `resources.diskMB` is rounded UP to the next GB. Defaults to 10.
*/
defaultVolumeGB?: number
/** Internal port the preview service forwards to (the Vite dev server). Defaults to 5173. */
previewPort?: number
/**
* Preview URL template. Placeholders `{app}`, `{machineId}` and `{port}` are
* all replaced globally. Defaults to `https://{app}.fly.dev`.
*
* For a control plane that runs on the SAME Fly 6PN, the private form
* `http://{machineId}.vm.{app}.internal:{port}` reaches the Machine with no
* public exposure at all — but it does NOT work across custom 6PNs, which is
* exactly what `appPerProject` creates. See the module `@remarks`.
*/
previewUrlTemplate?: string
/**
* Attach a public Fly Proxy service (`80` → http, `443` → tls+http) forwarding
* to `FlyioConfig.previewPort`. Defaults to `true`. Set `false` for a
* fully private sandbox reached only over 6PN/Flycast.
*/
publicService?: boolean
/**
* Fly Proxy idle behaviour for the preview service. Defaults to `'off'` —
* never idle the Machine — because the autostop timer counts the PROXY
* service's idle time, not the workload's, so a sandbox with nobody looking at
* its preview is suspended out from under a build that is still running.
* `suspend` is the scale-to-zero mapping (resume from a memory snapshot on the
* next request) and `stop` is a full stop with a cold boot on wake; opt into
* either deliberately.
*
* This is the provider's STEADY-STATE policy, reported as
* `SandboxProvider.defaultIdleAction`. A caller can override it for one sandbox
* with `SandboxConfig.idleAction` and restore this value afterwards with
* `Sandbox.setIdleAction` — which is what a production deploy does so its
* destination cannot suspend mid-copy (m103, 2026-10-17).
*/
autostop?: 'off' | 'stop' | 'suspend'
/**
* Allocate a shared Anycast IPv4 for a newly-created app so `<app>.fly.dev`
* serves traffic. Defaults to `false` (opt-in). See the module `@remarks` —
* the accepted `type` values for the IP-assignment endpoint are NOT enumerated
* in Fly's OpenAPI specification.
*/
assignSharedIpv4?: boolean
/** IP type requested when `FlyioConfig.assignSharedIpv4` is on. Defaults to `shared_v4`. */
ipAssignmentType?: string
/** Prefix for the Machine metadata keys this provider owns. Defaults to `molecule-sandbox`. */
metadataPrefix?: string
/** Timeout for a single Machines API request, in ms. Defaults to 30000. */
requestTimeoutMs?: number
/**
* TOTAL budget, in seconds, that `create()`/`start()`/`wake()` block waiting
* for the Machine to actually reach `started`. Fly's `GET .../wait` blocks
* for at most 60 seconds per call, so a larger budget is spent as consecutive
* wait rounds. Defaults to 180 — a Machine whose image is not yet cached on
* its host (every first boot after an image push) pulls it before starting,
* and that alone can exceed a single 60 s round.
*
* Without this wait, `start()` resolves the moment Fly ACCEPTS the request,
* and the caller's very next `exec` hits a Machine that is not running yet.
*/
startTimeoutSeconds?: number
/**
* Ports a sandbox Machine may open outbound connections on. Setting this makes
* the provider apply a Fly **network policy** to every app it provisions;
* leaving it unset applies no policy at all, and `verifyEgress()` will then
* observe (correctly) that egress is `open`. Falls back to
* `FLY_SANDBOX_EGRESS_ALLOWED_PORTS` (`tcp:3128,udp:53`).
*
* Read `https://fly.io/docs/machines/guides-examples/network-policies/`
* before choosing a value, because the mechanism is narrower than it looks:
* rules match protocol and port ONLY — no host, no CIDR, no ranges — so this
* can never be a host allowlist. `tcp:443` lets a sandbox reach EVERY host on
* the internet that listens on 443. To get a host allowlist, allow only the
* port of an egress proxy you control and route sandbox traffic through it.
*
* An empty array is rejected rather than treated as "deny all": Fly documents
* the deny default as a consequence of an `allow` rule existing, and says
* nothing about a rule with no ports.
*/
egressAllowedPorts?: FlyNetworkPolicyPort[]
/** Name of the egress policy this provider owns on each app. Defaults to `molecule-sandbox-egress`. */
egressPolicyName?: string
/**
* Apps every sandbox must be able to reach ACROSS its per-project 6PN — the
* tenant Postgres cluster and the control plane's egress proxy. For each one,
* this provider allocates a **Flycast** private address into the project's own
* network when the project's app is created, and releases it on `destroy()`.
* Falls back to `FLY_SANDBOX_PRIVATE_SERVICES` (`<app>:<port>` pairs).
*
* Without this, a project with a database DOES NOT WORK on Fly: per-project
// … truncated — the full declaration is in node_modules/@molecule/api-code-sandbox-flyio/dist/types.d.ts
ProcessEnv
Environment variables the Fly.io sandbox provider reads. Every one is
overridden by the matching FlyioConfig field.
interface ProcessEnv {
/** Fly API token. Also accepted as `FLY_ACCESS_TOKEN`. */
FLY_API_TOKEN?: string
/** Fly API token (flyctl's variable name). Used when `FLY_API_TOKEN` is unset. */
FLY_ACCESS_TOKEN?: string
/** Machines API base URL. `/v1` is appended when the value carries no path. */
FLY_API_HOSTNAME?: string
/** Fly organization slug owning the sandbox apps (default `personal`). */
FLY_ORG_SLUG?: string
/** Shared Fly app name, used only when `FLY_SANDBOX_APP_PER_PROJECT=false`. */
FLY_SANDBOX_APP?: string
/** Per-project Fly app name prefix (default `mol-sandbox`). */
FLY_SANDBOX_APP_PREFIX?: string
/** `false` disables per-project apps (refused in production). */
FLY_SANDBOX_APP_PER_PROJECT?: string
/** Fly region for Machines and volumes (default `iad`). */
FLY_REGION?: string
/** OCI image for sandbox Machines. */
FLY_SANDBOX_IMAGE?: string
/**
* Comma-separated `protocol:port` pairs a sandbox may open outbound
* connections on (e.g. `tcp:3128,udp:53`). Set it to make this provider apply
* a Fly network policy to every sandbox app; unset means no policy.
*/
FLY_SANDBOX_EGRESS_ALLOWED_PORTS?: string
/**
* Comma-separated `<fly-app>:<port>` pairs every sandbox must reach across its
* per-project 6PN, e.g. `molecule-pg-tenant:5432,molecule-api:3129`. Each one
* gets a Flycast private address allocated into the project's network at app
* creation and released on destroy. Overridden by `config.privateServices`.
*/
FLY_SANDBOX_PRIVATE_SERVICES?: string
/**
* Comma-separated literal `ip:port` targets the egress probe attempts. Shared
* with the Docker bond so a provider swap keeps the same configuration.
*/
SANDBOX_EGRESS_PROBE_TARGETS?: string
/** Per-connection timeout for the egress probe, in ms (default 3000). */
SANDBOX_EGRESS_PROBE_TIMEOUT_MS?: string
/** Bucket holding sandbox templates. Overridden by `config.templateBucket`. */
SANDBOX_TEMPLATE_BUCKET?: string
/** Bucket name as exported by `fly storage create`. Used when `SANDBOX_TEMPLATE_BUCKET` is unset. */
BUCKET_NAME?: string
/** S3 endpoint for the template bucket. Overridden by `config.templateEndpoint`. */
SANDBOX_TEMPLATE_ENDPOINT?: string
/** S3 endpoint as exported by `fly storage create`. Used when `SANDBOX_TEMPLATE_ENDPOINT` is unset. */
AWS_ENDPOINT_URL_S3?: string
/** Signing region for the template store (default `auto`). Overridden by `config.templateRegion`. */
SANDBOX_TEMPLATE_REGION?: string
/** Signing region, standard AWS variable. Used when `SANDBOX_TEMPLATE_REGION` is unset. */
AWS_REGION?: string
/** Access key for the template store. Overridden by `config.templateAccessKeyId`. */
SANDBOX_TEMPLATE_ACCESS_KEY_ID?: string
/** Access key, standard AWS variable (also what `fly storage create` sets). */
AWS_ACCESS_KEY_ID?: string
/** Secret key for the template store. Overridden by `config.templateSecretAccessKey`. */
SANDBOX_TEMPLATE_SECRET_ACCESS_KEY?: string
/** Secret key, standard AWS variable (also what `fly storage create` sets). */
AWS_SECRET_ACCESS_KEY?: string
/** Session token for the template store. Overridden by `config.templateSessionToken`. */
SANDBOX_TEMPLATE_SESSION_TOKEN?: string
/** Session token, standard AWS variable. */
AWS_SESSION_TOKEN?: string
/** Key prefix for template objects (default `molecule-sandbox-templates`). */
SANDBOX_TEMPLATE_PREFIX?: string
/** `true` addresses the template bucket path-style rather than virtual-host style. */
SANDBOX_TEMPLATE_FORCE_PATH_STYLE?: string
}
FlyPrivateService
One app made reachable from every sandbox's per-project 6PN through a Flycast private address.
See the flycast.ts module description for why this exists at all: apps on
separate 6PNs "can never communicate unless explicitly configured to do so",
and this declaration IS that explicit configuration.
interface FlyPrivateService {
/**
* The Fly app to allocate a Flycast address ON. The sandbox reaches it at
* `<app>.flycast`. It must already exist, must carry a services block, and
* must be in the same organization.
*/
app: string
/**
* The TCP port a sandbox dials it at. Unioned into the egress network policy,
* and checked against every `.flycast` URL in a sandbox's environment.
*/
port: number
}
FlyIpAssignment
An IP assignment, as returned by POST/GET /v1/apps/{app}/ip_assignments
(IPAssignment in https://docs.machines.dev/openapi.json).
Note what is NOT here: the assignment's network. Fly's own schema is
{created_at, ip, region, service_name, shared}, so a listing cannot tell you
which 6PN a private address serves — which is why this provider records the
addresses it allocated in the sandbox Machine's metadata instead of
rediscovering them.
interface FlyIpAssignment {
/** The allocated address. Required for a later `DELETE .../ip_assignments/{ip}`. */
ip?: string
/** Region the address was allocated in, when Fly reports one. */
region?: string
/** Service the address is bound to, when the assignment names one. */
service_name?: string
/** Whether the address is shared (Anycast v4) rather than dedicated. */
shared?: boolean
}
FlyNetworkPolicyPort
One protocol/port pair in a Fly network-policy rule.
Fly matches on protocol and port only — there is no host, IP, CIDR or port-range matching. See https://fly.io/docs/machines/guides-examples/network-policies/.
interface FlyNetworkPolicyPort {
/** `tcp` or `udp` — the only documented values. */
protocol: 'tcp' | 'udp'
/** A single port. Ranges are not supported. */
port: number
}
FlyNetworkPolicyRule
One rule in a Fly network policy. allow is the only documented action:
"Once you create a rule for a given direction, the default for that direction
becomes drop."
There is no destination field, and that is measured rather than assumed
(2026-08-16, live API): ipv6_cidrs, ipv4_cidrs, cidrs, destinations,
apps, app, to and dst were each POSTed on a rule and each came back
absent — the stored rule is {action, direction, ports} and nothing more. An
allow tcp/5432 rule therefore permits the Machine to reach port 5432 on
every host on the internet.
interface FlyNetworkPolicyRule {
/** Only `allow` is supported by Fly. */
action: 'allow'
/** Traffic direction the rule (and its implied deny default) applies to. */
direction: 'ingress' | 'egress'
/** Ports this rule permits. */
ports: FlyNetworkPolicyPort[]
}
FlyNetworkPolicySelector
Which Machines in the app a policy applies to — the SOURCE side only. There is
no destination selector: an apps array is the one candidate the API answers
about, and it answers 400 {"error":"apps array not currently supported in selectors"} (measured 2026-08-16); every other candidate field is silently
dropped. Documented criteria combine with AND, so this provider uses
{ all: true } alone.
The API's LIST response returns this object under the key netpolSelector
while accepting it as selector on write — which is why nothing here reads it
back by name.
interface FlyNetworkPolicySelector {
/** Match every Machine in the app. */
all?: boolean
/** Match specific Machine ids. */
machines?: Array<{ id: string }>
/** Match Machines carrying these metadata keys. */
metadata?: Record<string, string>
}
FlyNetworkPolicy
A Fly network policy, as accepted by
POST /v1/apps/{app}/network_policies. The endpoint is documented in Fly's
guide and announcement but is NOT in its OpenAPI specification, so only the
fields those two documents show are modelled.
interface FlyNetworkPolicy {
/** Present to UPDATE an existing policy; omitted to create one. */
id?: string
/** Policy name, unique per app in practice. */
name: string
/** Machines the policy applies to. */
selector: FlyNetworkPolicySelector
/** The allow rules. */
rules: FlyNetworkPolicyRule[]
}
FlyMachine
A Fly Machine, as returned by the Machines API (fields this provider reads).
interface FlyMachine {
id: string
name?: string
/** Optional so a response that omits it degrades to `created` rather than crashing the mapper. */
state?: FlyMachineState
region?: string
private_ip?: string
config?: FlyMachineConfig
/** RFC 3339 creation time, as Fly reports it. */
created_at?: string
/**
* Lifecycle events, newest first. The ONLY place Fly records when a Machine
* was first started — there is no `started_at` field — which is what
* distinguishes a stopped Machine from one that was created and never ran.
*/
events?: FlyMachineEvent[]
}
FlyMachineEvent
One entry of FlyMachine.events.
interface FlyMachineEvent {
/** `start`, `launch`, `exit`, … */
type?: string
status?: string
/** Epoch milliseconds. */
timestamp?: number
}
FlyApp
An app as returned by GET /v1/apps?org_slug=… (the fields this provider reads).
interface FlyApp {
id?: string
name: string
machine_count?: number
}
FlyAppList
Response body of GET /v1/apps?org_slug=….
interface FlyAppList {
/** Total apps in the org, which the returned page may not cover. */
total_apps?: number
apps?: FlyApp[]
}
FlyOrgMachine
A Machine as returned by the org-wide list endpoint (GET /orgs/{org}/machines).
interface FlyOrgMachine extends FlyMachine {
app_name: string
}
FlyMachineConfig
Machine configuration (fly.MachineConfig) — the subset this provider writes.
interface FlyMachineConfig {
image: string
env?: Record<string, string>
metadata?: Record<string, string>
guest?: { cpus?: number; cpu_kind?: string; memory_mb?: number }
mounts?: Array<{ volume?: string; name?: string; path: string }>
services?: FlyMachineService[]
restart?: { policy?: 'no' | 'always' | 'on-failure' | 'spot-price'; max_retries?: number }
auto_destroy?: boolean
init?: { exec?: string[]; entrypoint?: string[]; cmd?: string[]; tty?: boolean }
}
FlyMachineService
A Fly Proxy service attached to a Machine (fly.MachineService).
interface FlyMachineService {
protocol: string
internal_port: number
ports: Array<{ port: number; handlers?: string[]; force_https?: boolean }>
autostart?: boolean
autostop?: 'off' | 'stop' | 'suspend'
min_machines_running?: number
}
FlyVolume
A Fly volume (fields this provider reads).
interface FlyVolume {
id: string
name: string
state?: string
size_gb?: number
region?: string
attached_machine_id?: string
}
FlyExecResponse
Response body of POST /v1/apps/{app}/machines/{id}/exec (flydv1.ExecResponse).
interface FlyExecResponse {
stdout?: string
stderr?: string
exit_code?: number
exit_signal?: number
}
Types
RawExec
Runs one exec call against a Machine. Injected so this module stays transport-free.
type RawExec = (command: string[], timeoutSeconds: number) => Promise<FlyExecResponse>
FlyMachineState
A Fly Machine state, as documented at https://fly.io/docs/machines/machine-states/.
Modelled as a union of the documented values plus string so an unrecognized
future state is carried through rather than crashing the mapper.
type FlyMachineState =
| 'created'
| 'creating'
| 'starting'
| 'started'
| 'stopping'
| 'stopped'
| 'suspending'
| 'suspended'
| 'restarting'
| 'updating'
| 'replacing'
| 'replaced'
| 'migrated'
| 'destroying'
| 'destroyed'
| 'failed'
| 'launch_failed'
| (string & {})
Classes
FlyApiError
An error from the Fly Machines API. Carries the HTTP status and the raw
response body so callers can branch on 404 (absent) or 409 (conflict)
without string-matching a message.
class FlyApiError extends Error {
/** HTTP status code, or 0 for a transport-level failure (timeout, DNS, reset). */
readonly status: number
/** Raw response body text, truncated to a sane length for logging. */
readonly body: string
/** HTTP method of the failed request. */
readonly method: string
/** API path of the failed request (no base URL). */
readonly path: string
/**
* Creates a Fly API error.
* @param method - HTTP method of the failed request.
* @param path - API path of the failed request.
* @param status - HTTP status code, or 0 for a transport failure.
* @param body - Raw response body text.
* @param options - Standard error options (used to carry a `cause`).
*/
constructor(
method: string,
path: string,
status: number,
body: string,
options?: { cause?: unknown },
) {
super(
t(
'codeSandbox.flyio.error.apiError',
{ method, path, status: String(status), error: body },
{ defaultValue: `Fly API ${method} ${path}: ${status} ${body}` },
),
options,
)
this.name = 'FlyApiError'
this.status = status
this.body = body
this.method = method
this.path = path
}
}
FlyApiClient
Minimal Fly Machines API client: bearer auth, JSON bodies, per-request
timeout, bounded retries with backoff, and account-wide request pacing via
paceFlyRequest.
class FlyApiClient {
private readonly token: () => string | undefined
private readonly baseUrl: string
private readonly timeoutMs: number
private readonly fetchImpl: typeof fetch
/**
* Injectable delay, shared with the provider: a retry that a CALLER drives
* (the provider's exec-precondition backoff) waits on the same injected clock
* the client's own retries use, so tests collapse every wait in one place.
*/
readonly sleep: (ms: number) => Promise<void>
/**
* Creates a Fly Machines API client.
* @param options - Token resolver, base URL, and injectable fetch/sleep.
*/
constructor(options: FlyApiClientOptions) {
this.token = options.token
this.baseUrl = options.baseUrl
this.timeoutMs = options.timeoutMs ?? DEFAULT_REQUEST_TIMEOUT_MS
this.fetchImpl = options.fetchImpl ?? globalThis.fetch
this.sleep =
options.sleep ?? ((ms: number) => new Promise<void>((resolve) => setTimeout(resolve, ms)))
}
/**
* Issues a request against the Machines API and parses the JSON response.
*
* @template T - Expected shape of the parsed response body.
* @param path - API path relative to the base URL, e.g. `/apps/foo/machines`.
* @param options - Method, body, timeout, attempts, and statuses to null on.
* @returns The parsed body, or `null` for an empty body or a status listed in
* `options.nullOn`.
* @throws {FlyApiError} On a non-2xx status that is not listed in `nullOn`,
* or when every retryable attempt is exhausted.
*/
async request<T>(path: string, options: FlyRequestOptions = {}): Promise<T | null> {
const method = options.method ?? 'GET'
const attempts = options.attempts ?? DEFAULT_ATTEMPTS
const token = this.token()
if (!token) {
throw new Error(
t('codeSandbox.flyio.error.noToken', undefined, {
defaultValue:
'No Fly API token — set FLY_API_TOKEN (or FLY_ACCESS_TOKEN), or pass config.apiToken.',
}),
)
}
let lastError: FlyApiError | undefined
for (let attempt = 1; attempt <= attempts; attempt++) {
// Pace EVERY network call (including retries) through the process-wide
// limiter so a boot's burst of exec calls never trips Fly's account 429.
await paceFlyRequest(this.sleep)
const controller = new AbortController()
const timer = setTimeout(() => controller.abort(), options.timeoutMs ?? this.timeoutMs)
// Initialized to 0 ("no answer received"), which is the value that
// stands when the request throws before a response arrives.
let status = 0
let retryAfter: string | null = null
try {
const response = await this.fetchImpl(`${this.baseUrl}${path}`, {
method,
headers: {
Authorization: `Bearer ${token}`,
...(options.body === undefined ? {} : { 'Content-Type': 'application/json' }),
},
...(options.body === undefined ? {} : { body: JSON.stringify(options.body) }),
signal: controller.signal,
})
status = response.status
retryAfter = response.headers?.get?.('retry-after') ?? null
const text = await response.text()
if (status >= 200 && status < 300) {
if (!text.trim()) return null
try {
return JSON.parse(text) as T
} catch (error) {
// A 2xx with a non-JSON body is a real protocol surprise (an
// intercepting proxy, an HTML error page behind a 200). Refusing it
// loudly beats handing the caller a silently-empty result.
throw new FlyApiError(method, path, status, text.slice(0, 500), { cause: error })
}
}
if (options.nullOn?.includes(status)) return null
lastError = new FlyApiError(method, path, status, text.slice(0, 500))
} catch (error) {
if (error instanceof FlyApiError) throw error
// Transport-level failure (abort/timeout, DNS, connection reset) — no
// status was ever received, so it is modelled as status 0 and retried.
lastError = new FlyApiError(
method,
path,
0,
error instanceof Error ? error.message : String(error),
{ cause: error },
)
} finally {
clearTimeout(timer)
}
const retryable =
isRetryableStatus(status) || (options.retryStatuses?.includes(status) ?? false)
if (!retryable || attempt === attempts) break
const delay = retryDelayMs(attempt, retryAfter)
logger.warn(`Fly API transient failure on ${method} ${path} — retrying`, {
attempt,
attempts,
status,
delay,
})
await this.sleep(delay)
}
throw lastError ?? new FlyApiError(method, path, 0, 'unknown failure')
}
}
Functions
isRetryableStatus(status)
Decides whether a failed attempt is worth repeating.
429 (the documented per-action rate limit) and 5xx are transient — the
request was well-formed and the same call can succeed moments later. Every
other 4xx is a real answer (no such app, name taken, bad token) and
retrying it only wastes the rate-limit budget that the retry exists to
protect. Status 0 means the transport failed before any answer arrived.
function isRetryableStatus(status: number): boolean
status— HTTP status, or 0 for a transport-level failure.
Returns: true when the request should be retried.
retryDelayMs(attempt, retryAfter)
Resolves how long to wait before the next attempt.
Honors a Retry-After header when the server sent one (Fly returns it on
429), clamped to MAX_RETRY_DELAY_MS; otherwise uses exponential
backoff from a 500 ms base. Fly's documented budget is one request per second
per action, so the first backoff step deliberately exceeds a second.
function retryDelayMs(attempt: number, retryAfter?: string | null): number
attempt— The 1-based attempt number that just failed.retryAfter— RawRetry-Afterheader value, if present.
Returns: The delay in milliseconds before the next attempt.
normalizeApiUrl(value)
Normalizes a configured API base into a URL prefix ending in /v1.
FLY_API_HOSTNAME is documented as a bare host (https://api.machines.dev,
or http://_api.internal:4280 from inside a Fly private network), so a value
with no path gets /v1 appended. A value that already carries a path is used
verbatim, which is how an operator points at a proxy or a pinned version.
function normalizeApiUrl(value: string | undefined): string
value— Configured base URL, or undefined.
Returns: The base URL with no trailing slash.
resetFlyPacerForTests()
Test-only: reset the shared pacer so timing tests start from a clean slate.
function resetFlyPacerForTests(): void
formatEgressProbeTarget(target)
Renders a target the way it is written in configuration, so an operator can paste it straight back.
function formatEgressProbeTarget(target: EgressProbeTarget): string
target— The parsed target.
Returns: host:port, with IPv6 hosts bracketed.
parseEgressProbeTargets(raw)
Parses probe targets from configuration.
Only LITERAL addresses are accepted, never hostnames — Fly's own troubleshooting guidance for testing a policy is to "use direct IP addresses (not hostnames) to test blocked traffic to avoid DNS masking", and a probe that failed at name resolution would report a blocked connection it never actually attempted.
function parseEgressProbeTargets(raw: string | string[] | undefined): EgressProbeTarget[]
raw— Comma-separatedip:portstring, or an array of them. IPv6 literals must be bracketed ([2606:4700:4700::1111]:443).
Returns: The valid targets, in order. Invalid entries are dropped rather than throwing: the caller turns an empty result into an inconclusive verdict, which is the honest outcome for "nothing to probe".
parseEgressAllowedPorts(raw)
Parses an allowed-port list for the egress network policy.
An EMPTY list is deliberately not expressible. Fly documents the deny default
as a consequence of at least one allow rule existing for a direction; what
an allow rule with zero ports does is not documented anywhere, so a provider
that sent one would be guessing at a security control. An operator who wants
near-total denial allows a single port nothing in the sandbox uses.
function parseEgressAllowedPorts(raw: string | undefined): FlyNetworkPolicyPort[] | undefined
raw— Comma-separatedprotocol:portpairs, e.g.tcp:3128,udp:53. A bare port is treated as TCP.
Returns: The parsed ports, or undefined when the input is absent or empty (meaning "apply no policy"), which is NOT the same as "deny everything".
buildEgressProbeCommand(targets, timeoutMs)
Builds the shell command that observes egress from inside a Machine.
Raw net.connect on purpose: an HTTP client would honour proxy environment
variables and report the PROXY's policy instead of the network's. Every target
is attempted in parallel and the process exits
EGRESS_PROBE_EXIT_REACHED if any connected,
EGRESS_PROBE_EXIT_BLOCKED if all were refused or timed out.
function buildEgressProbeCommand(targets: EgressProbeTarget[], timeoutMs: number): string
targets— Literal targets to attempt.timeoutMs— Per-connection timeout.
Returns: A sh-safe command string.
describePublicReach(policyPorts)
States, in the verdict itself, what an allowed port actually permits.
A Fly network policy has no destination (see the module description — measured
against the live API, not inferred), so every allowed port is open to every
host on the internet, not only to the private service it was derived for. That
is the single most consequential thing about this mechanism and it was invisible
everywhere it mattered: the probe attempts one port the policy denies, reports
filtered, and an operator reasonably reads that as "nothing gets out".
Naming it here means the residual is restated on every re-probe (the consumer re-verifies every 15 minutes) instead of living only in a document.
function describePublicReach(policyPorts: FlyNetworkPolicyPort[] | undefined): string
policyPorts— Allowed ports, orundefinedwhen no policy is applied.
Returns: A sentence naming the public reach, or '' when there is nothing to say (no policy, or nothing but UDP/53 — DNS is a documented residual of its own and not a TCP channel).
unexpectedPolicyPorts(applied, intended)
Finds ports the policy Fly holds allows that this provider never configured.
The raw-connect probe can only speak for the targets it was handed, so a
policy widened outside this provider — edited by hand to unblock something, or
left behind by an older build — is invisible to it: the probe's own port stays
denied and the verdict stays filtered while a port nothing here asked for is
open to the whole internet. Comparing what Fly holds against what this
provider intended is the one check that can catch that, and it is why the
verdict reads the policy back rather than trusting the write.
function unexpectedPolicyPorts(
applied: FlyNetworkPolicyPort[] | undefined,
intended: FlyNetworkPolicyPort[] | undefined,
): FlyNetworkPolicyPort[]
applied— Ports in the policy Fly actually holds, orundefinedwhen the readback found nothing (in which case there is nothing to compare).intended— Ports this provider configured, orundefinedwhen it applies no policy.
Returns: The applied ports with no counterpart in intended, in order.
verdictForProbeExit(exitCode, context)
Maps a probe's exit status onto an EgressVerdict.
The mapping is the whole security contract of this file, so it is total and has exactly one path to each state:
| Exit | Verdict | Meaning |
|---|---|---|
EGRESS_PROBE_EXIT_REACHED | open | A raw socket reached a public IP. |
EGRESS_PROBE_EXIT_BLOCKED | filtered | Every attempt was refused or timed out. |
| anything else | inconclusive | The probe did not complete its attempts. |
Every other outcome — a missing interpreter (127), a killed process, Fly
returning no status at all (-1), an API failure before the probe ever ran — is
inconclusive. Collapsing any of those into filtered is precisely the "I
could not look" ⇒ "I looked and it is safe" conflation the capability exists
to prevent, and the caller refuses to boot production on anything but
filtered.
function verdictForProbeExit(exitCode: number, context: EgressProbeContext): EgressVerdict
exitCode— The probe process's exit status.context— App, targets and the policy this provider applied.
Returns: The verdict.
extractPolicyId(response, name)
Finds an existing policy's id in a LIST response, so the next write updates it instead of stacking duplicates.
Neither the guide nor the announcement specifies what
GET /v1/apps/{app}/network_policies/ returns, and the endpoint is absent
from Fly's OpenAPI specification — so this accepts a bare array or an object
wrapping one under any of the plausible keys, and returns undefined for
anything it does not recognize. undefined is safe: the caller then POSTs
without an id, which still applies the policy.
function extractPolicyId(response: unknown, name: string): string | undefined
response— The raw parsed LIST response.name— The policy name to match.
Returns: The matching policy's id, or undefined.
extractPolicyPorts(response, name)
Reads back the egress ports the policy Fly ACTUALLY holds allows.
The write is not the observation: this provider POSTs a policy, but what
governs a Machine is whatever Fly has when that Machine boots — which may
differ because a person, another tool or an older build changed it. Reading it
back is what lets unexpectedPolicyPorts catch a policy wider than the
one this provider configured, so verifyEgress() can fail on drift its raw
connects were never going to attempt.
Parsed as defensively as extractPolicyId, and for the same reason:
neither Fly document specifies the LIST response shape. The live API answers
with a bare array whose selector key is netpolSelector rather than the
selector it accepts (verified 2026-08-16), which is exactly why nothing here
depends on any key but name and rules.
function extractPolicyPorts(response: unknown, name: string): FlyNetworkPolicyPort[] | undefined
response— The raw parsed LIST response.name— The policy name to match.
Returns: The allowed egress ports, or undefined when no policy of that name is present or the shape is unrecognized. undefined means "nothing to compare", never "nothing is allowed".
toExecResult(response)
Normalizes a Fly exec response into the core ExecResult.
Fly reports exit_signal separately from exit_code; a signalled process has
no meaningful exit code, so it is rendered the way a POSIX shell does, as
128 + signal.
function toExecResult(response: FlyExecResponse): ExecResult
response— The raw Fly exec response.
Returns: The normalized result.
buildScript(command, opts, defaultCwd)
Builds the shell script body for a command: change directory, export the requested environment, then run the command.
function buildScript(command: string, opts: ExecOptions | undefined, defaultCwd: string): string
command— The shell command to run.opts— Optional cwd and environment.defaultCwd— Directory used whenopts.cwdis unset.
Returns: The script source, newline separated.
execCommand(rawExec, command, opts, defaultCwd)
Executes a command on a Machine, choosing the direct or detached strategy based on the caller's time budget.
A command too large to pass inline (over FLY_EXEC_MAX_ARG_BYTES) is
spilled to a file via chunked writes and run from there, so there is no size
ceiling the caller must respect.
function execCommand(
rawExec: RawExec,
command: string,
opts: ExecOptions | undefined,
defaultCwd: string,
): Promise<ExecResult>
rawExec— Transport callback issuing one Fly exec call.command— The shell command to run.opts— Core exec options (cwd, env, timeout in ms).defaultCwd— Working directory used whenopts.cwdis unset.
Returns: The command's stdout, stderr and exit code.
flycastHost(app)
Renders the DNS name a sandbox dials to reach an app over its Flycast address.
function flycastHost(app: string): string
app— The target Fly app name.
Returns: <app>.flycast.
parsePrivateServices(raw)
Parses the declared cross-network services.
Each entry is <app>:<port> — the Fly app to allocate a Flycast address on,
and the TCP port a sandbox dials it at. The port is not cosmetic: it is what
mergeEgressPorts adds to the Fly network policy, and what
assertPrivateRoutesForEnv checks the injected connection URLs against.
function parsePrivateServices(raw: string | undefined): FlyPrivateService[] | undefined
raw— Comma-separatedapp:portpairs, e.g.molecule-pg-tenant:5432,molecule-api:3129.
Returns: The declared services, deduplicated, in order — or undefined when the input is absent or empty, meaning "allocate nothing", which is correct for a control plane that shares one 6PN with its sandboxes.
mergeEgressPorts(ports, services)
Adds every declared private-service port to the egress network policy.
A Fly network policy is deny-by-default once any rule exists for a direction,
so a sandbox told to dial molecule-pg-tenant.flycast:5432 under a policy
allowing only tcp:3128 would fail to connect with no diagnostic beyond a
timeout. Deriving the port from the declaration — rather than asking the
operator to keep two lists in step — is what makes that class of outage
impossible: FLY-OPERATOR-SETUP.md § 9 can keep saying "never widen this list",
because the operator never has to.
It is deliberately a UNION and never a replacement, so an operator's list is still exactly what they wrote plus the ports their own declarations require, and the applied policy is logged.
The addition is load-bearing — VERIFIED 2026-08-16, not inferred. Fly states
"Network policies only apply to traffic directly to and from Machines. They do
not affect traffic routed through the Fly Proxy"
(https://fly.io/docs/machines/guides-examples/network-policies/), and Flycast
traffic IS routed through Fly Proxy, so it was an open question whether these
ports did anything. Measured on a throwaway app with a Flycast address into
its own 6PN: under a policy allowing only udp:53, molecule-pg-tenant.flycast
and molecule-api.flycast both RESOLVED and every TCP connect to them was
dropped; re-applying the policy with tcp:443 opened 443 and nothing else.
Fly's sentence is about INGRESS — a Machine's egress TOWARD a Flycast address
(description truncated — read the full text in node_modules/@molecule/api-code-sandbox-flyio/dist/flycast.d.ts.)
function mergeEgressPorts(
ports: FlyNetworkPolicyPort[] | undefined,
services: FlyPrivateService[] | undefined,
): FlyNetworkPolicyPort[] | undefined
ports— The operator's configured ports, orundefinedwhen no policy is being applied at all.services— The declared private services, orundefined.
Returns: The union, deduplicated — or undefined when ports is undefined. "No policy" is never turned INTO a policy here: applying one that Fly then uses to drop everything else is not something to infer from an unrelated setting.
encodePrivateRoutes(routes)
Encodes allocated addresses for storage in Machine metadata.
function encodePrivateRoutes(routes: Record<string, string>): string
routes— Target app name → allocated private address.
Returns: app=address pairs joined by commas. An IPv6 literal contains neither , nor =, so the encoding is unambiguous.
decodePrivateRoutes(raw)
Decodes the addresses recorded by encodePrivateRoutes.
function decodePrivateRoutes(raw: string | undefined): Record<string, string>
raw— The metadata value, orundefinedwhen the Machine carries none.
Returns: Target app name → allocated private address. Unparseable entries are skipped: this value only drives cleanup and self-healing, and a malformed entry must not stop either.
assertPrivateRoutesForEnv(env, services)
Refuses to boot a sandbox that is told to dial a Flycast host this provider did not allocate a route for.
This is the check that turns the product-breaking failure into a startup
error. The control plane bakes DATABASE_URL (and the proxy environment) into
every sandbox; on Fly those must name <app>.flycast, and that name resolves
ONLY because this provider allocated a private address into the project's
network for that app. Get the two out of step — a SANDBOX_DB_HOST pointing
at an app nobody declared, a port that does not match the one the policy
allows — and the sandbox boots healthy, the scaffolded app cannot connect, and
the only symptom is a connection timeout inside someone else's project.
It is also the guard on the control-plane cluster: reaching
molecule-pg-control.flycast from a sandbox would require an operator to have
declared it as a private service, which is an explicit act with its own name
in the configuration — never something this provider arranges on its own.
function assertPrivateRoutesForEnv(
env: Record<string, string> | undefined,
services: FlyPrivateService[] | undefined,
): void
env— The environment the caller is baking into the sandbox.services— The declared private services, orundefined.
toSandboxId(app, machineId)
Builds the opaque sandbox id addressing one Machine inside one app.
function toSandboxId(app: string, machineId: string): string
app— Fly app name.machineId— Fly Machine id.
Returns: The composite sandbox id.
parseSandboxId(id)
Splits a composite sandbox id back into its app and Machine id.
function parseSandboxId(id: string): { app: string; machineId: string }
id— A sandbox id previously produced bytoSandboxId.
Returns: The app name and Machine id.
appNameForProject(prefix, projectId)
Derives a Fly app name for a project.
Fly app names are globally unique DNS labels: lowercase alphanumerics and
hyphens, no leading/trailing hyphen, at most 63 characters (they become
<app>.fly.dev). Project ids are uuids, so <prefix>-<uuid> fits with room
to spare; longer ids are truncated rather than rejected, and the trailing
hyphen that truncation can leave is stripped.
function appNameForProject(prefix: string, projectId: string): string
prefix— Configured app-name prefix.projectId— The project id fromSandboxConfig.
Returns: A syntactically valid Fly app name.
mapMachineState(state)
Maps a Fly Machine state onto the core Sandbox['status'] union.
The core has four statuses and Fly documents seventeen states (https://fly.io/docs/machines/machine-states/), so the mapping is lossy by construction:
started→running.suspended/suspending→sleeping. This is the mapping this bond exists for:sleep()suspends andwake()resumes from the memory snapshot.- In-flight transitions (
creating,starting,restarting,updating,replacing,migrated) →creating, i.e. "not usable yet, will be". - Everything else — including
failedandlaunch_failed— →stopped. The core union has no error status, so a failed Machine is reported as stopped;failedis distinguishable only viaisFailedState.
An unrecognized future state falls through to stopped, which is the safe
default: a caller retries a start rather than assuming a usable sandbox.
function mapMachineState(state: FlyMachineState): 'creating' | 'stopped' | 'running' | 'sleeping'
state— Raw Fly Machine state string.
Returns: The core sandbox status.
isFailedState(state)
Reports whether a Fly Machine state means the Machine errored, rather than
having been stopped on purpose. mapMachineState flattens both to
stopped because the core union has no failure status, so this is the only
way to tell them apart.
function isFailedState(state: FlyMachineState): boolean
state— Raw Fly Machine state string.
Returns: true for failed and launch_failed.
isTransientExecPrecondition(error, machineState)
Whether a failed exec is Fly's TRANSIENT precondition — one a repeat (after a short wait) can clear, rather than a Machine state the exec has to wait out.
Fly answers the exec endpoint with 412 in two different situations, and only the BODY and the Machine's own reported state separate them:
failed_precondition: exec request failed: EOF— the request reached the Machine's exec agent and the connection dropped before the command ran. Always transient: the Machine isstartedand its agent recovers in seconds.failed_precondition: machine not running— transient ONLY while Fly's own Machines API reports the Machinestarted,startingorreplacing(machineState). The exec endpoint lags the state Fly reports: a Machine whose config or image was just updated, or that is seconds past boot, readsstartedinGET /machines/{id}while its exec endpoint still refuses requests. Against a Machine Fly reportsstopped/suspended/destroyedthe same body is the REAL precondition — the Machine is not running, andFlyioSandboxProvider.startMachineis what answers it; repeating the exec would only burn the caller's budget.
function isTransientExecPrecondition(error: unknown, machineState?: string | null): boolean
error— The failure an exec attempt raised.machineState— The stateGET /machines/{id}reports for the Machine, ornull/undefinedwhen it could not be read (which classifies case 2 as NOT transient — an unknown state is never assumed recoverable).
Returns: true when repeating the exec is the right recovery.
isMachineWarmingUp(state)
Whether a state Fly reports for a Machine means the Machine is up, or on its
way up — the states in which an exec that was refused for machine not running is the exec endpoint lagging the state, not the truth.
A deliberate allow-list rather than "anything but stopped/suspended": the
three states here are the ones a refused exec was observed to recover from,
and every other state — stopped, suspended, destroyed, failed,
created, unreadable — keeps today's immediate throw. Widening it without
that evidence would turn a real precondition into a retry budget spent
against a Machine that is not coming up.
function isMachineWarmingUp(state: string | null | undefined): boolean
state— The Machine state fromGET /machines/{id}, if readable.
Returns: true when waiting and repeating the exec can clear the refusal.
createProvider(config, client, store)
Creates a Fly.io Machines sandbox provider.
function createProvider(
config?: FlyioConfig,
client?: FlyApiClient,
store?: ObjectStore | null,
): SandboxProvider
config— Optional Fly configuration: API token/URL, org slug, app naming and isolation mode, region, base image, guest sizing, preview service, and the preview URL template. Every field has an env fallback.client— Injectable Machines API client, for tests.store— Injectable template object store, for tests. Resolved from the template settings (or their env fallbacks) on first use when omitted.
Returns: A SandboxProvider backed by Fly Machines.
resolveTemplateStorage(config)
Resolves template-storage settings from config and the environment.
Returns null when the store is not configured at all, which the template
methods turn into an actionable error naming the settings. Partial
configuration is treated as unconfigured for the same reason: half a
connection cannot be used, and reporting it as a connection failure would
point an operator at the network instead of at their settings.
function resolveTemplateStorage(config: FlyioConfig): ResolvedStorage | null
config— Fly provider configuration.
Returns: The resolved settings, or null when storage is not configured.
createTemplateStore(config)
Creates the object store used for templates, or null when it is not
configured.
function createTemplateStore(config: FlyioConfig): ObjectStore | null
config— Fly provider configuration; every field has an env fallback.
Returns: An ObjectStore, or null when no bucket/credentials are set.
assertTemplateId(templateId)
Validate a caller-supplied template id.
function assertTemplateId(templateId: string): void
templateId— The caller's identifier.
assertCapturePath(path)
Validate a path that will be interpolated into a sh -c command AND used as a
tar member selector.
function assertCapturePath(path: string): void
path— Candidate absolute path inside the sandbox.
archiveMember(path)
Converts an absolute capture path into the relative member name used in the
archive, so an archive of /workspace holds workspace/… and extracts with
tar -C /.
function archiveMember(path: string): string
path— Absolute capture path.
Returns: The path with its leading and trailing slashes removed.
manifestKey(store, templateId)
Key of a template's manifest object.
function manifestKey(store: ObjectStore, templateId: string): string
archiveKey(store, templateId)
Key of a template's archive object.
function archiveKey(store: ObjectStore, templateId: string): string
leaseKey(store, templateId, leaseId)
Key of one in-flight restore lease.
function leaseKey(store: ObjectStore, templateId: string, leaseId: string): string
templateRef(store, templateId)
The provider-native reference for a template. OPAQUE to callers.
function templateRef(store: ObjectStore, templateId: string): string
store— The configured object store.templateId— The caller's identifier.
Returns: An s3://bucket/key reference to the archive.
requireStore(ctx)
Require a configured object store, naming the settings that turn it on.
Thrown rather than reported as "no such template": the capability is present — Fly plus any S3-compatible endpoint — and only the address is missing, so an operator needs to see the configuration error rather than watch every boot quietly rebuild from scratch.
function requireStore(ctx: TemplateContext): ObjectStore
ctx— Template context.
Returns: The configured store.
buildCaptureCommand(paths, url, maxBytes)
Builds the in-sandbox capture command: archive the named paths, refuse an oversized result, and upload it to a presigned URL.
The archive is written to a file BEFORE the upload rather than piped into
curl, for two reasons that both bite: a pipeline reports only the last
command's status, so a failed tar would upload a truncated archive and
report success; and an upload from a pipe has no Content-Length, so curl
sends Transfer-Encoding: chunked, which S3 rejects on a presigned PUT.
function buildCaptureCommand(paths: string[], url: string, maxBytes: number): string
paths— Absolute capture paths, already validated.url— PresignedPUTURL for the archive object.maxBytes— Largest archive that may be uploaded.
Returns: A sh script.
buildRestoreCommand(paths, url)
Builds the in-sandbox restore command: download the archive, extract ONLY the manifest's paths, and prove no setuid/setgid file survived.
See the module docs for why each flag is here. The download is to a file
rather than piped into tar so a failed download cannot be reported as a
successful extraction of a truncated stream.
function buildRestoreCommand(paths: string[], url: string): string
paths— Absolute capture paths from the CONTROL-PLANE manifest.url— PresignedGETURL for the archive object.
Returns: A sh script.
hasLiveLease(leases, ttlMs, now)
Decides whether any lease means a restore is still in flight.
A lease with no readable timestamp counts as LIVE. That is the whole rule the core states for this field: an unreadable answer resolves to in-use, never to free.
function hasLiveLease(leases: StoredObject[], ttlMs: number, now: number): boolean
leases— Lease objects for one template.ttlMs— How long a lease can possibly correspond to a live restore.now— Current time in ms since the epoch.
Returns: true when at least one lease is live.
readTemplate(ctx, templateId)
Read one template, and the manifest behind it, by the caller's identifier.
Separate from getTemplate because the restore path needs the
manifest's capturePaths — the extraction surface — and the public shape does
not carry them.
function readTemplate(
ctx: TemplateContext,
templateId: string,
): Promise<{ template: SandboxTemplate; manifest: TemplateManifest } | null>
ctx— Template context.templateId— The caller's identifier.
Returns: The template and its manifest, or null when no usable template exists.
getTemplate(ctx, templateId)
Read one template by the caller's identifier.
function getTemplate(ctx: TemplateContext, templateId: string): Promise<SandboxTemplate | null>
ctx— Template context.templateId— The caller's identifier.
Returns: The template, or null when no template has that id.
listTemplates(ctx, options)
Enumerate templates so the caller can apply its retention policy.
function listTemplates(
ctx: TemplateContext,
options?: ListTemplatesOptions,
): Promise<SandboxTemplate[]>
ctx— Template context.options— Narrowing by id prefix.
Returns: Every matching template.
commitTemplate(ctx, options)
Capture a sandbox's filesystem into a reusable template.
function commitTemplate(
ctx: TemplateContext,
options: CommitTemplateOptions,
): Promise<SandboxTemplate>
ctx— Template context.options— What to capture and what to call it.
Returns: The template as it now exists.
removeTemplate(ctx, templateId)
Delete a template, refusing while a restore is still reading it.
function removeTemplate(ctx: TemplateContext, templateId: string): Promise<void>
ctx— Template context.templateId— The caller's identifier.
acquireRestoreLease(ctx, templateId, leaseId)
Records that a restore of this template is in flight.
The lease lives in the store rather than in this process, so a second control plane enforcing a retention budget can see it. A control plane that dies mid-restore leaves the lease behind; it stops counting once it is older than the transfer budget, because no live restore can outlast that budget.
function acquireRestoreLease(
ctx: TemplateContext,
templateId: string,
leaseId: string,
): Promise<string>
ctx— Template context.templateId— The caller's identifier.leaseId— Unique id for this restore.
Returns: The lease's key, for release.
releaseRestoreLease(ctx, key)
Releases a restore lease.
Best-effort: the restore has already finished, and a stale lease only delays an eviction until it ages out. Failing the boot over it would trade a completed sandbox for a bookkeeping error.
function releaseRestoreLease(ctx: TemplateContext, key: string): Promise<void>
ctx— Template context.key— The lease key returned byacquireRestoreLease.
shellQuote(value)
Wraps a string in single quotes, escaping any single quotes it contains.
Every command this provider runs goes through sh -c, so an unquoted path or
value is a command-injection vector. Double quotes are NOT sufficient — $(),
backticks and ! still expand inside them.
function shellQuote(value: string): string
value— The raw string to embed in a shell command.
Returns: A single-quoted, shell-safe token.
renderEnvExports(env)
Renders ExecOptions.env as export statements to prepend to a script.
Fly's exec endpoint takes no environment map (only command, stdin,
container and timeout), so per-command environment has to be set inside
the shell script itself.
function renderEnvExports(env: Record<string, string> | undefined): string[]
env— Environment variables to set for the command.
Returns: One export NAME='value' line per variable, in insertion order.
Constants
DEFAULT_EGRESS_POLICY_NAME
Default name of the egress policy this provider owns on each sandbox app.
const DEFAULT_EGRESS_POLICY_NAME: 'molecule-sandbox-egress'
EGRESS_PROBE_EXIT_REACHED
Probe exit status meaning at least one raw connection to a public IP SUCCEEDED. Deliberately not 1, so an interpreter error (which exits 1) can never be mistaken for a completed observation.
const EGRESS_PROBE_EXIT_REACHED: 9
EGRESS_PROBE_EXIT_BLOCKED
Probe exit status meaning every raw connection attempt was refused or timed out.
const EGRESS_PROBE_EXIT_BLOCKED: 0
DEFAULT_EGRESS_PROBE_TARGETS
Default probe targets: one IPv4 and one IPv6 literal, both anycast resolvers that answer on 443 from anywhere. IPv6 is included because Fly Machines have native public IPv6 egress and "often egress over IPv6 when the destination has an AAAA record" (https://fly.io/docs/networking/egress-ips/) — probing only IPv4 could miss an open v6 path entirely.
const DEFAULT_EGRESS_PROBE_TARGETS: '1.1.1.1:443,[2606:4700:4700::1111]:443'
FLY_EXEC_MAX_TIMEOUT_SECONDS
Fly's hard ceiling on the exec endpoint's timeout field, in seconds.
Verified against Fly staff guidance (see the module docs); a larger value is
rejected by the API rather than clamped.
const FLY_EXEC_MAX_TIMEOUT_SECONDS: 60
DIRECT_EXEC_BUDGET_SECONDS
Largest budget still run as a single direct exec, in seconds. Kept below
FLY_EXEC_MAX_TIMEOUT_SECONDS so the API's own timeout is never the
thing that fires first — the caller's timeout should be.
const DIRECT_EXEC_BUDGET_SECONDS: 55
MAX_CAPTURED_OUTPUT_BYTES
Cap on captured stdout/stderr per detached command, in bytes. The exec
response carries output inline as JSON, so an unbounded cat of a build log
would try to materialize the whole thing in one response body.
const MAX_CAPTURED_OUTPUT_BYTES: number
FLY_EXEC_MAX_ARG_BYTES
Hard cap on the sh -c <script> argument SENT to Fly's exec endpoint. This is
NOT the OS MAX_ARG_STRLEN (128 KB) — the Fly Machines exec API rejects a
command well below that, and does so INVISIBLY: it answers 200 with
exit_code: 0 and an Unhandled rejection: Rejection([PayloadTooLarge…])
body, so the command never runs yet reads as success. Measured against the
live API: ~15.8 KB scripts run, ~23.7 KB are rejected. 15 KB is the safe
ceiling; callers that move bulk data (writeFile, importFiles) chunk beneath it.
const FLY_EXEC_MAX_ARG_BYTES: 15000
INDETERMINATE_EXIT_CODE
Exit code reported when a command's outcome is genuinely unknown — it was still running when the caller's budget expired, or Fly returned no status. Distinct from every real shell exit status (0-255), matching the Docker provider's convention so callers can tell "unfinished" from "failed".
const INDETERMINATE_EXIT_CODE: -1
FLYCAST_SUFFIX
DNS suffix Fly serves a Flycast address at: "If an app has a Flycast address
allocated to it, there will be an AAAA record at my-app-name.flycast"
(https://fly.io/docs/networking/flycast/).
const FLYCAST_SUFFIX: '.flycast'
PRIVATE_ROUTES_METADATA_SUFFIX
Machine-metadata key suffix (after the configured prefix) recording the
private addresses allocated for a project, so destroy() can release exactly
the ones it created. See consequence 2 in the module description: a listing on
the target app cannot tell you which network an address serves.
const PRIVATE_ROUTES_METADATA_SUFFIX: 'privateRoutes'
DEFAULT_PRIVATE_IP_TYPE
type sent to POST /apps/{app}/ip_assignments for a Flycast address. This
is the value flyctl passes for fly ips allocate-v6 --private; the field has
no enum in Fly's OpenAPI specification, so it is configurable rather than
hardcoded.
const DEFAULT_PRIVATE_IP_TYPE: 'private_v6'
provider
The provider implementation.
const provider: SandboxProvider
DEFAULT_TEMPLATE_PREFIX
Default key prefix for every object this bond writes.
const DEFAULT_TEMPLATE_PREFIX: 'molecule-sandbox-templates'
DEFAULT_TEMPLATE_REGION
Region sent when none is configured. Tigris (and most S3-compatible stores)
accept auto; the AWS SDK requires some region to build a signature, so this
is not optional even against a store that ignores it.
const DEFAULT_TEMPLATE_REGION: 'auto'
MAX_PRESIGN_EXPIRY_SECONDS
AWS's documented ceiling on a SigV4 presigned URL: "If you use the AWS CLI or AWS SDKs, the expiration time can be set as high as 7 days." (https://docs.aws.amazon.com/AmazonS3/latest/userguide/using-presigned-url.html) A larger value is not rejected at signing time — it simply produces a URL S3 refuses — so it is clamped here.
const MAX_PRESIGN_EXPIRY_SECONDS: number
MAX_ARCHIVE_BYTES
Single-PUT ceiling: "With a single PUT operation, you can upload a single
object up to 5 GB in size"
(https://docs.aws.amazon.com/AmazonS3/latest/userguide/upload-objects.html).
The sandbox uploads with one presigned PUT, so an archive above this cannot be
stored and the capture refuses before spending the bandwidth.
const MAX_ARCHIVE_BYTES: number
DEFAULT_TRANSFER_TIMEOUT_MS
Default wall-clock budget for a capture or restore transfer, in ms.
const DEFAULT_TRANSFER_TIMEOUT_MS: 900000
DEFAULT_PRESIGN_EXPIRY_SECONDS
Default lifetime of a presigned capture/restore URL, in seconds.
const DEFAULT_PRESIGN_EXPIRY_SECONDS: 3600
LEASE_GRACE_MS
Extra time a restore lease counts as live beyond the transfer budget, covering
the Machine boot and the API round trips that bracket the transfer itself.
A lease older than transferTimeoutMs + this cannot belong to a live restore,
because the restore's own exec is bounded by that budget.
const LEASE_GRACE_MS: 120000
EXIT_ARCHIVE_TOO_LARGE
Exit status the capture script uses when the archive is over the size ceiling.
const EXIT_ARCHIVE_TOO_LARGE: 90
EXIT_SETUID_SURVIVED
Exit status the restore script uses when a setuid/setgid file survived the sweep.
const EXIT_SETUID_SURVIVED: 91
EXIT_TAR_FAILED
Exit status the capture script uses when tar failed outright (as opposed to warning).
const EXIT_TAR_FAILED: 92
TAR_CHANGED_MARKER
Marker the capture script prints on stderr when tar reported changed files.
const TAR_CHANGED_MARKER: 'MOL_TAR_FILES_CHANGED'
Core Interface
Implements @molecule/api-code-sandbox interface.
Bond Wiring
Setup function to register this provider with the core interface:
import { setProvider } from '@molecule/api-code-sandbox'
import { provider } from '@molecule/api-code-sandbox-flyio'
export function setupCodeSandboxFlyio(): void {
setProvider(provider)
}
Injection Notes
Requirements
Peer dependencies:
@molecule/api-bond^1.0.1@molecule/api-code-sandbox^1.2.0@molecule/api-i18n^1.0.1
Environment Variables
FLY_API_TOKEN(required) — Fly.io API token- Setup: Run
fly tokens create org <org>(orfly tokens deployfor a single app). Needs permission to create apps, Machines and volumes in the org. - Get it here: https://fly.io/docs/machines/api/working-with-machines-api/
- Example:
FlyV1 fm2_...
- Setup: Run
FLY_ORG_SLUG(required) — Fly organization slug- Setup: The organization that owns the sandbox apps —
fly orgs listshows the slug. - Get it here: https://fly.io/dashboard
- Example:
personal
- Setup: The organization that owns the sandbox apps —
FLY_REGION(optional) — Fly region — default:iad- Setup: Region for sandbox Machines and their volumes.
- Get it here: https://fly.io/docs/reference/regions/
- Example:
iad
FLY_SANDBOX_IMAGE(optional) — Sandbox OCI image — default:registry.fly.io/molecule-sandbox:latest- Setup: Image Fly pulls for each sandbox Machine. Must live in a registry Fly can reach — a local image is invisible to it.
- Example:
registry.fly.io/molecule-sandbox:latest
FLY_API_HOSTNAME(optional) — Machines API base URL — default:https://api.machines.dev- Setup: Override the Machines API endpoint. From inside a Fly private network use http://_api.internal:4280.
- Example:
https://api.machines.dev
FLY_SANDBOX_EGRESS_ALLOWED_PORTS(optional) — Sandbox egress allow-list (protocol:port)- Setup: Comma-separated protocol:port pairs a sandbox may open outbound connections on. Setting it makes the provider apply a Fly network policy to every sandbox app, after which Fly drops all other egress. Unset means NO egress restriction and verifyEgress() will report
open. Fly matches protocol and port only — there is no host or CIDR matching — so allow only the port of an egress proxy you control if you need a host allow-list. Ports declared in FLY_SANDBOX_PRIVATE_SERVICES are added automatically — do not widen this list by hand. - Get it here: https://fly.io/docs/machines/guides-examples/network-policies/
- Example:
tcp:3129,udp:53
- Setup: Comma-separated protocol:port pairs a sandbox may open outbound connections on. Setting it makes the provider apply a Fly network policy to every sandbox app, after which Fly drops all other egress. Unset means NO egress restriction and verifyEgress() will report
FLY_SANDBOX_PRIVATE_SERVICES(optional) — Cross-6PN services (app:port)- Setup: Comma-separated <fly-app>:<port> pairs every sandbox must reach across its per-project 6PN — typically the tenant Postgres cluster and the control plane's egress proxy. The provider allocates a Flycast private address on each app INTO the project's own network at boot and releases it on destroy; the sandbox then dials <app>.flycast:<port>. Without this, a project with a database DOES NOT WORK on Fly: apps on separate 6PNs cannot communicate. Each target app needs an [http_service] or [services] section and must bind 0.0.0.0. The declared port is added to the egress network policy automatically.
- Get it here: https://fly.io/docs/networking/flycast/
- Example:
molecule-pg-tenant:5432,molecule-api:3129
Runtime Dependencies
@aws-sdk/client-s3@aws-sdk/s3-request-presigner@molecule/api-bond@molecule/api-code-sandbox@molecule/api-i18n@molecule/api-proxy-agent
Sources. Every endpoint, payload field and documented behaviour used here
was checked against Fly's own material, not recalled:
the OpenAPI specification (https://docs.machines.dev/openapi.json,
servers: https://api.machines.dev/v1), the Machines resource reference
(https://fly.io/docs/machines/api/machines-resource/), Working with the
Machines API (https://fly.io/docs/machines/api/working-with-machines-api/),
Machine states (https://fly.io/docs/machines/machine-states/), Suspend and
Resume (https://fly.io/docs/reference/suspend-resume/), Custom private
networks (https://fly.io/docs/networking/custom-private-networks/), Private
networking (https://fly.io/docs/networking/private-networking/), Fly Volumes
(https://fly.io/docs/volumes/overview/), Network Policies
(https://fly.io/docs/machines/guides-examples/network-policies/ and its
announcement https://community.fly.io/t/new-feature-network-policies/19173),
Egress IP addresses (https://fly.io/docs/networking/egress-ips/), rate limits
(https://fly.io/docs/machines/api/working-with-machines-api/), and the exec
timeout ceiling
(https://community.fly.io/t/extending-timeout-of-execute-command-machines-api-endpoint/26074).
For the template capability: Fly Volume snapshots
(https://fly.io/docs/volumes/snapshots/), Tigris object storage on Fly
(https://fly.io/docs/tigris/) and its S3 API coverage
(https://www.tigrisdata.com/docs/api/s3/), presigned URLs
(https://docs.aws.amazon.com/AmazonS3/latest/userguide/using-presigned-url.html),
the 5 GB single-PUT ceiling
(https://docs.aws.amazon.com/AmazonS3/latest/userguide/upload-objects.html),
and the AWS SDK's checksum defaults
(https://docs.aws.amazon.com/sdkref/latest/guide/feature-dataintegrity.html).
The tar behaviours the restore's containment rests on — a .. member
refused, a leading / stripped, a write through a symlink refused, and
setuid/setgid dropped by --no-same-permissions — were verified by running
GNU tar 1.35 against crafted archives rather than inferred from its manual.
Absence vs failure: get() only answers null for a REAL absence.
get() (and each read shaped like it — list(), find(), getTemplate())
reserves null/[] for a positive not-found answer — Fly answered 404 for
the app or the Machine — and THROWS for every other failure: 5xx, 429 past
the client's retries, timeouts, network errors. A caller that reconciles
from null (clearing sandbox ids, reaping Machines) must never read a
transient Fly API blip as "the sandbox does not exist"; treat a rejection
from these methods as "unknown", never as "gone". The thrown error carries
the underlying FlyApiError (HTTP status, or 0 for a transport failure) as
its cause.
sleep() is suspend, not stop — that is the point of this bond.
sleep() calls POST /v1/apps/{app}/machines/{id}/suspend, which "uses
Firecracker snapshots to capture the entire VM state: CPU registers, memory
contents, open file handles"; wake() calls start, which restores it —
"Resume from suspend: a few hundred ms" versus "Cold start: ~2+ seconds".
Fly documents that "Suspended machines cost the same as stopped machines:
storage only. There are no CPU/RAM charges." Two caveats that bite: suspend is
discouraged above 2 GB of guest memory, and calling stop() on a SUSPENDED
Machine invalidates its snapshot, forcing a cold boot on the next wake() —
so do not "tidy up" a sleeping sandbox with a stop.
exec is capped at 60 seconds by Fly, and this provider works around it.
The exec endpoint rejects a timeout above 60 s outright. Any command whose
budget exceeds that is therefore run DETACHED inside the Machine — script file
plus redirected stdout/stderr plus an exit-status file — and polled to
completion with short execs. The caller still gets one ExecResult. The
observable differences: output is not streamed, and a command still running
when the budget expires returns exitCode: -1 (indeterminate) with whatever
output existed at that moment, exactly like the Docker bond's convention.
Captured output is capped at 5 MB per stream.
spawn() is NOT implemented. It is optional in the core interface, and
Fly's exec is strictly request/response — no connection upgrade, no streaming
stdout, and stdin only as a single up-front string. There is no honest way to
back a SpawnHandle with it, so the method is absent rather than faked.
Callers must feature-detect (sandbox.spawn?.(…)), which the interface
already requires.
onFileChange() is NOT implemented — it registers nothing and returns a
real no-op unsubscribe, and warns once. The Machines API exposes no
filesystem-event channel, and the only alternative is polling through the
rate-limited exec endpoint, which would spend the whole API budget to deliver
changes seconds late. Poll readDir/readFile yourself if you need it.
Tenant isolation: one app per project, each on its own 6PN. Every Machine
in a Fly app shares one 6PN private network and can reach every other Machine
in that app by private IPv6 — the same cross-tenant exposure the Docker bond's
shared bridge network has. So by default this provider creates one Fly app
per project (<prefix>-<projectId>) with network set to a custom 6PN, which
Fly documents as the tenant-isolation mechanism: "Apps on separate 6PNs can
never communicate unless explicitly configured to do so." Shared-app mode
(appPerProject: false) exists for single-tenant deployments and is REFUSED
in production. An app's network cannot be changed after creation, so an app
that already exists is used as-is rather than silently "fixed".
Egress: verifyEgress() OBSERVES, and Fly's only lever is ports. 6PN
isolation answers "can app A reach app B", not "can a sandbox reach the
internet". For that, the one mechanism Fly documents is a network policy —
app-scoped allow rules where "Once you create a rule for a given direction,
the default for that direction becomes drop." Set egressAllowedPorts (or
FLY_SANDBOX_EGRESS_ALLOWED_PORTS=tcp:3128,udp:53) and this provider applies
one to every app it provisions, BEFORE creating the Machine — Fly applies a
policy at boot ("restart or redeploy the Machines for changes to take
effect"), so a policy that lands afterwards does not cover the running
Machine. A failure to apply it FAILS the boot rather than quietly starting an
unfiltered sandbox. Three limits are load-bearing and cannot be designed
around: rules match protocol and port only (no host, no CIDR, no ranges),
so a policy can never be a host allowlist — tcp:443 lets a sandbox reach
every host on the internet that listens on 443; policies do not apply to Fly
Proxy traffic; and the endpoints, while documented in the guide and the
announcement, are absent from Fly's OpenAPI specification, so the LIST
response shape is parsed defensively. To get host-level control, allow only
the port of an egress proxy you run and route sandbox traffic through it.
Crossing the 6PN: privateServices is what makes a database work. The
per-project 6PN that stops tenants reaching each other also stops a sandbox
reaching the tenant Postgres cluster and the control plane's egress proxy —
both of which the product REQUIRES. Fly documents three ways across a 6PN
boundary (https://fly.io/docs/networking/custom-private-networks/): a public
service IP, fly-replay, and a Flycast private address allocated into the
calling network. Only the third grants ONE directed edge with nothing exposed
publicly, so it is what this bond implements. Declare the targets —
privateServices: [{ app: 'molecule-pg-tenant', port: 5432 }], or
FLY_SANDBOX_PRIVATE_SERVICES=molecule-pg-tenant:5432,molecule-api:3129 — and
create() allocates one address per target INTO the project's own network
(POST /apps/{target}/ip_assignments, the REST form of
fly ips allocate-v6 --private --network <project-net>), records them in the
Machine's metadata, and destroy() releases them. The sandbox then dials
<target>.flycast:<port>.
Four properties of that are worth knowing before you deploy it:
- Isolation is unchanged. An allocation is a one-way grant from ONE project's network to ONE shared app. No sandbox app ever receives one, so no sandbox becomes reachable from anywhere it was not already.
- The declared port is unioned into the egress network policy, because a
policy is deny-by-default once any rule exists — declaring
molecule-pg-tenant:5432under atcp:3128policy would drop every database connection in the fleet. Whether the union is even load-bearing is UNVERIFIED: Fly says policies "do not affect traffic routed through the Fly Proxy" and Flycast IS Fly Proxy, but says nothing about a Machine's EGRESS toward a Flycast address. An inert entry costs a port; a missing one costs the fleet. - Any
.flycastURL in a sandbox's environment must have a matching declaration, orcreate()throws. That check is what turns "the route was silently never created" — a healthy sandbox that cannot reach its own database — into a startup error, and it is also the guard on the control-plane cluster: reaching it would require an operator to declare it by name. - The target app needs an
[http_service]/[services]section and must bind0.0.0.0(https://fly.io/docs/networking/flycast/). This bond cannot do that for you. Fly documents no limit on how many private addresses one app may hold, in either direction, so the shared target apps accumulate one address per live project — reconcile withfly ips list --app <target>.
verifyEgress() then PROVES the result instead of reporting the
configuration: it boots a throwaway Machine through the same ensureApp path
every sandbox uses, attempts raw TCP connects to literal public IPs with proxy
environment blanked, and maps connected → open, refused/timed-out →
filtered, anything else → inconclusive. It costs one Machine boot (and, in
per-project mode, one app create/delete) per call, so cache the verdict.
The preview URL is a template, and the isolation choice constrains it.
getPreviewUrl() renders previewUrlTemplate (default
https://{app}.fly.dev) with {app}, {machineId} and {port}. The
private form http://{machineId}.vm.{app}.internal:{port} reaches a Machine
over 6PN with no public exposure at all — but only from the SAME 6PN, which
per-project apps deliberately prevent. So a control plane that wants private
previews must reach each app another way (Flycast, or a public router app
using fly-replay), and a control plane off Fly entirely must use the public
form. Public serving additionally requires an allocated IP: set
assignSharedIpv4: true to request one at app-creation time. The accepted
type values for POST /v1/apps/{app}/ip_assignments are NOT enumerated in
Fly's OpenAPI specification — shared_v4 is this provider's default and is
configurable via ipAssignmentType, but treat that value as UNVERIFIED.
Volumes are provisioned by create(), not by createVolume(). The
optional core methods createVolume/removeVolume/volumeExists are
deliberately NOT implemented: they receive only a name, while a Fly volume
needs an app, a region and a size, and a volume created in the wrong app can
never be mounted. So passing SandboxConfig.volumeName provisions the volume
inside the project's app, mounted at /workspace; resources.diskMB is
rounded UP to whole GB. Fly documents that "a volume can be attached to only
one Machine", which matches one sandbox per project. Fly does not document the
legal character set for volume names, so names are conservatively reduced to
[A-Za-z0-9_] — an assumption, flagged as such.
Warm start: templates are tar archives in object storage. Fly cannot
commit a running Machine — it PULLS images from a registry — and the
Machines API has no file-transfer endpoint at all, so a template here is a
tar.gz of the caller-named capturePaths in an S3-compatible bucket
(Tigris on Fly: fly storage create exports exactly the variables this bond
reads). The bytes move DIRECTLY between the sandbox and the store over
presigned URLs — the control plane only handles the URL and the manifest —
because pushing hundreds of megabytes through exec's JSON response is not a
slower transfer, it is a different order of magnitude. commitTemplate
REQUIRES capturePaths: unlike Docker, where the container's own image
carries the filesystem, the archive here IS the template, so committing none
would store an empty template that later boots successfully into an empty
workspace. Configure it with templateBucket + templateAccessKeyId +
templateSecretAccessKey (+ templateEndpoint for a non-AWS store); without
them the template methods throw an actionable error naming the settings
rather than pretending the capability is absent, the same call the Docker
bond makes for templateRegistry.
The volume-snapshot alternative was checked and rejected on Fly's own
documentation, not on preference: POST /v1/apps/{app}/volumes does accept
snapshot_id and source_volume_id, but "Every Fly Volume belongs to a Fly
App and you can't share a volume between apps"
(https://fly.io/docs/volumes/overview/) — and this bond gives every project
its own app. A snapshot could serve a per-project restore point and never the
cross-project warm start the capability exists for.
Booting from a template FAILS when the template is gone, rather than
falling back to the base image, which would hand back a healthy-looking
sandbox whose filesystem is not the one that was named. One wording in the
core cannot be honored literally: templateId does NOT override image
here, because a Fly template is a filesystem rather than an image — the image
still selects the OS and toolchain, and the template supplies the captured
paths on top of it. A restore that fails DESTROYS the Machine before throwing.
The tenant boundary is the restore, not the capture. A capture runs inside
a sandbox the tenant controls, so nothing done there is a security control.
The restore runs in a fresh Machine before anyone holds a handle to it, and
extracts ONLY the paths recorded in the control-plane-written manifest (they
are tar member selectors), with --no-same-owner --no-same-permissions,
followed by a find -perm /6000 -exec chmod a-s sweep and then a second
find that FAILS the restore if any setuid/setgid file survived. GNU tar's
own refusals — a .. member, an absolute member, a write through a symlink —
all exit non-zero and therefore fail the restore too.
inUse means "a restore is in flight", and cannot mean more than that. On
Docker a template is the image a container runs, so deleting it destroys the
container. Here the archive is COPIED into the sandbox at boot and never
referenced again, so removing a template cannot destroy a running sandbox —
the only window is a restore still reading it, which is tracked with a lease
object in the same store (so a second control plane sees it) and ages out
after the transfer budget. A lease whose timestamp cannot be read counts as
live, and any failure to LIST or read the store throws rather than reporting
"absent" or "not in use".
publishTemplate/fetchTemplate are NOT implemented, deliberately. They
exist for providers whose templates are host-local; object storage is already
the shared store, so there is nothing to publish or fetch.
importFiles IS implemented; exportFiles is not. Import takes the same
route the template restore does — a presigned object-store URL the Machine
pulls itself, falling back to chunked base64 over exec when no store is
configured — so an app tree can be delivered into a fresh Machine. There is no
inverse: Fly's only read channel is the exec endpoint's JSON response, and a
base64 emulation of a whole-tree export would look supported and route every
byte through the control plane. Pair a Fly destination with a source that CAN
export (the core's rooting contract makes them interchangeable).
Sandbox ids are composite: "<app>:<machineId>". A Fly Machine id is only
unique within its app and every endpoint is /v1/apps/{app}/machines/{id}, so
a bare Machine id cannot be addressed. Keep treating the id as opaque.
The image must be pullable BY FLY. Fly pulls the image itself, so a local
molecule-sandbox:latest is invisible to it — push to the org's
registry.fly.io repository or a public registry first. Unlike the Docker
bond, nothing here can pre-pull on a host you control.
Rate limits are handled in the transport, but they also shape throughput.
Fly documents the Machines API at "1 request, per second, per action … with a
short-term burst limit up to 3 req/s, per action", scoped "per-action,
per-machine … That might be Machine ID or App ID, depending on the type of
request", with Get Machine at 5 req/s and app deletions capped at 100 per
minute. 429 and 5xx are retried with Retry-After-aware backoff, plus the
404 an app-propagation lag answers the Machine create with; every other
4xx is a real answer and is never retried. The consequence to plan for is
that EVERY file operation is an exec API call, so one sandbox's reads and
writes serialize at roughly one per second and a multi-megabyte writeFile
(chunked at ~45 KB decoded) takes tens of seconds. An agent that edits files
in a tight loop will feel this; batch where you can.
A refused exec that dropped the connection is retried; a refused exec that
names a Machine state is not. Fly answers 412 failed_precondition: exec request failed: EOF when the request reached a started Machine's exec agent
and the connection died before the command ran — a Machine seconds past boot,
or Fly's own exec burst. exec retries that one at the transport (up to four
attempts, ~11 s of backoff, each preceded by a Machine-state read and a wait
for started when it has not got there), because every caller's alternative
is to throw away a whole boot or deploy and start over. The OTHER 412,
failed_precondition: machine not running, is a real state and is not
repeated: start()/wake() wait it out before the first exec. A 412 that is
not retried comes back with the Machine's current state appended to its
message, and a 412 the retries could not clear carries the state it ended in.
create() launches the Machine, and start()/wake() WAIT for it. Unlike
Docker's create-then-start, the Machines API starts a Machine as part of
creation. POST .../start only means Fly accepted the request, so
start()/wake() then block on GET .../wait?state=started (Fly blocks
server-side for up to 60 s, tunable with startTimeoutSeconds) and throw if
the Machine never runs — without that, the caller's next exec would land on
a Machine that is still booting.
Statuses are lossy. Fly documents seventeen Machine states and the core
has four. failed/launch_failed both map to stopped because the core
union has no error status.
No per-sandbox disk quota beyond the volume. resources.diskMB sizes the
mounted volume; without volumeName it is not applied at all, and the rootfs
size is Fly's default.