@molecule/api-ai-decisions-jeff

Jeff decisions provider for molecule.dev — typed decisions from the open-weights Jeff models on a jeff-serve host you run.

Jeff (github.com/firelex/jeff; code MIT, weights Apache-2.0) is a family of fine-tunes of Qwen3.5-0.8B, Qwen3.5-2B and Gemma 4 E2B that answer choice, score and yes/no questions with a probability per option — no text generation. Its maker reports ~22 ms per request for the 0.8B on an NVIDIA RTX PRO 6000 and ~28 ms on an Apple M4 Max. jeff-serve speaks TypeSafe Jev's /v1/systemone protocol, so swapping to @molecule/api-ai-decisions-jev or -laya is a one-line change.

Auto-generated, AI-first package reference for the molecule.dev ecosystem. It is written to be read by coding agents as much as by people, and is generated from this package's source — edit src/index.ts JSDoc, not this file.

Quick Start

# Run the model server (NVIDIA GPU, or Apple silicon with JEFF_BACKEND=mlx)
git clone https://github.com/firelex/jeff 
uv hf download mstrasser/Jeff-Qwen3.5-0.8B --local-dir checkpoints/jeff-0.8b
JEFF_API_KEY=change-me JEFF_CHECKPOINT=checkpoints/jeff-0.8b PORT=8000 uv run jeff-serve

import { setProvider, requireProvider } from '@molecule/api-ai-decisions'
import { provider } from '@molecule/api-ai-decisions-jeff'

setProvider(provider) // reads JEFF_URL + JEFF_API_KEY on first use

const { answers } = await requireProvider().decide({
  state: 'My card was charged twice, please refund one of them',
  questions: {
    queue: {
      type: 'choice',
      instructions: 'Which team?',
      criteria: { billing: 'charges, refunds', tech: 'bugs, login' },
    },
    refund: { type: 'yesNo', instructions: 'The customer wants a refund.' },
  },
})
answers.queue.choice // 'billing'
answers.refund.probability // 0.95

Type

provider

Installation

npm install @molecule/api-ai-decisions-jeff @molecule/api-ai-decisions @molecule/api-secrets

API

Interfaces

JeffConfig

Configuration for the Jeff decisions provider.

interface JeffConfig {
  /** Base URL of the `jeff-serve` host. Defaults to `JEFF_URL`, then `http://localhost:8000`. */
  baseUrl?: string
  /** Bearer token, when the server sets `JEFF_API_KEY`. Defaults to the `JEFF_API_KEY` env var. */
  apiKey?: string
  /**
   * Extra request headers, resolved before each call and merged over the
   * defaults — for a host whose auth expires or is not a bearer token (a Cloud
   * Run ID token, Modal proxy auth). Pair with `@molecule/api-model-hosting`:
   * `headers: () => hosting.authHeaders(endpoint.id)`.
   */
  headers?: () => Record<string, string> | Promise<Record<string, string>>
  /**
   * Model id sent on every request. `jeff-serve` REQUIRES one and accepts only
   * `'jeff'`, `'jeff-latest'` or its own name (e.g. `'jeff-qwen3.5-0.8b'`) —
   * it answers with whichever checkpoint it loaded. Defaults to `'jeff-latest'`.
   */
  model?: string
  /**
   * Most `choice` options the loaded checkpoint answers (the server's `/health`
   * reports it as `max_options`). Defaults to 26 — the Jeff models released on
   * 2026-09-28. Raise it only for a checkpoint trained on more.
   */
  maxOptions?: number
}

WireQuestion

One question as sent on the wire.

interface WireQuestion {
  type: 'choice' | 'score' | 'noul'
  instructions: string
  criteria?: Record<string, string> | string[]
}

WireAnswer

One answer as received on the wire (only the fields we read).

interface WireAnswer {
  type?: string
  choice?: string
  score?: number
  noul?: number
  probabilities?: Record<string, number>
}

WireResponse

The response body (only the fields we read).

interface WireResponse {
  model?: string
  answers?: Record<string, WireAnswer>
  usage?: { input_tokens?: number; output_tokens?: number }
}

PostOptions

Options for postSystemOne.

interface PostOptions {
  /** Full endpoint URL. */
  url: string
  /** Bearer token, if any. */
  apiKey?: string
  /** Extra headers resolved before the request (merged over the defaults). */
  headers?: () => Record<string, string> | Promise<Record<string, string>>
  /** Request body. */
  body: Record<string, unknown>
  /** Abort signal. */
  signal?: AbortSignal
  /** Label for error messages (`'Jeff'`). */
  label: string
  /** Max retries on a retryable status (default 3). */
  maxRetries?: number
}

Functions

createProvider(config)

Creates a Jeff decisions provider.

function createProvider(config?: JeffConfig): AIDecisionsProvider

• config — Base URL, API key, model id and option limit.

Returns: An AIDecisionsProvider backed by a jeff-serve host.

toWireQuestions(questions)

Converts the core's questions into wire questions (yesNo → noul).

function toWireQuestions(questions: Record<string, DecisionQuestion>): Record<string, WireQuestion>

• questions — The questions keyed by id.

Returns: The wire questions object.

fromWireAnswer(id, question, wire, minConfidence)

Converts one wire answer into the core's answer for question.

function fromWireAnswer(
  id: string,
  question: DecisionQuestion,
  wire: WireAnswer | undefined,
  minConfidence?: number,
): DecisionAnswer

• id — The question id (for error messages).
• question — The question that was asked.
• wire — The wire answer.
• minConfidence — Optional low-confidence threshold.

Returns: The typed answer.

fromWireResponse(questions, body, minConfidence)

Converts a whole wire response into the core's result.

function fromWireResponse(questions: Q, body: WireResponse, minConfidence?: number): DecideResult<Q>

• questions — The questions that were asked.
• body — The parsed response body.
• minConfidence — Optional low-confidence threshold.

Returns: The typed result.

postSystemOne(opts)

POSTs a /v1/systemone request with retry on 429/5xx-busy, honouring Retry-After. Throws an Error carrying status on a non-2xx response.

function postSystemOne(opts: PostOptions): Promise<WireResponse>

• opts — Request options.

Returns: The parsed response body.

Constants

DEFAULT_JEFF_URL

jeff-serve's default bind (PORT defaults to 8000; its README examples use 8765).

const DEFAULT_JEFF_URL: 'http://localhost:8000'

DEFAULT_JEFF_MODEL

A model id every jeff-serve accepts; it answers with the checkpoint it loaded.

const DEFAULT_JEFF_MODEL: 'jeff-latest'

DEFAULT_JEFF_MAX_OPTIONS

Most choice options the released Jeff checkpoints answer (options are coded A–Z).

const DEFAULT_JEFF_MAX_OPTIONS: 26

JEFF_SCORE_LEVELS

jeff-serve's bounds on a score question's levels.

const JEFF_SCORE_LEVELS: { readonly min: 2; readonly max: 10 }

JEFF_MAX_IMAGES

Most images jeff-serve accepts per request.

const JEFF_MAX_IMAGES: 4

JEFF_IMAGE_TYPES

Image media types jeff-serve decodes.

const JEFF_IMAGE_TYPES: readonly string[]

provider

The provider implementation — lazy, so env vars are read on first use.

const provider: AIDecisionsProvider

aiDecisionsJeffSecretDefinitions

Secret definitions used by the Jeff decisions bond.

const aiDecisionsJeffSecretDefinitions: SecretDefinition[]

Core Interface

Implements @molecule/api-ai-decisions interface.

Bond Wiring

Setup function to register this provider with the core interface:

import { setProvider } from '@molecule/api-ai-decisions'
import { provider } from '@molecule/api-ai-decisions-jeff'

export function setupAiDecisionsJeff(): void {
  setProvider(provider)
}

Injection Notes

Requirements

Peer dependencies:

• @molecule/api-ai-decisions ^1.1.0
• @molecule/api-secrets ^1.0.1

Environment Variables

• JEFF_URL (optional) — Jeff server URL
  • Setup: The base URL of your jeff-serve host (clone github.com/firelex/jeff, uv sync, download a checkpoint, then JEFF_CHECKPOINT=<dir> uv run jeff-serve). Defaults to http://localhost:8000.
  • Get it here: https://github.com/firelex/jeff#readme
  • Example: http://localhost:8000
• JEFF_API_KEY (optional) — Jeff server API key
  • Setup: The bearer token your jeff-serve host requires, if you set JEFF_API_KEY on the server.
  • Get it here: https://github.com/firelex/jeff#readme
  • Example: a-long-random-string

Runtime Dependencies

• @molecule/api-ai-decisions

• @molecule/api-secrets

• You run the model. JEFF_URL (default http://localhost:8000) points at a jeff-serve host. Its code defaults to port 8000 but its README examples use PORT=8765 — set JEFF_URL to match. jeff-serve binds 127.0.0.1 unless you set JEFF_HOST. Set JEFF_API_KEY on BOTH sides to require a bearer token; without it anyone who can reach the server can use it.

• model is required on the wire and is NOT a checkpoint picker. The server accepts only 'jeff', 'jeff-latest' (the default here) or its own name, rejects anything else with a 422, and always answers with the ONE checkpoint it loaded (JEFF_CHECKPOINT). Never pass a Jev model id. Run a second server to use a second checkpoint.

• At most 26 choice options with the released checkpoints (options are coded A–Z; the server refuses more). score takes 2–10 levels. This bond refuses both before sending — shortlist long option lists first. Raise createProvider({ maxOptions }) only for a checkpoint whose /health reports a higher max_options.

• One request at a time. A busy server answers 529 + Retry-After: 1 and a loading one 503; this bond retries 429/502/503/504/529 up to 3 times. Errors carry the HTTP status.

• English text only. Use @molecule/api-ai-decisions-laya's multilingual checkpoint for other languages.

• Images: up to 4 PNG/JPEG/WebP images are forwarded, but only the PyTorch backend reads them (/health → modalities); the Apple MLX backend is text-only, and the maker publishes no image accuracy figures. For image questions prefer @molecule/api-ai-decisions-intern-decision.

• Small models don't reason. Expect fast choices between options you describe, not multi-step logic (the 0.8B scores 68% on BIG-Bench Hard vs Jev's 94%). Describe each option's consequences in plain words.

• Calibrate on your own data. The checkpoints carry one fitted temperature from the maker's training mix; recalibrate on labelled samples of your inputs and gate on minConfidence. See the core's remarks.

• Hosted on rented compute? Pass headers: () => hosting.authHeaders(endpoint.id) (@molecule/api-model-hosting) when the host's auth expires or is not a bearer token — e.g. a Cloud Run ID token or Modal proxy auth. It runs before every request and is merged over the defaults.

• Use the core's setProvider, not bond('ai-decisions', …) directly.

E2E Tests

Integration checklist — drive the real UI (live preview, no mocks), adapt each item to this app's actual screens/flows, and check every box off one by one. A box you can't check is an integration bug to fix — not a skip:

• Each flow that makes a decision (routing, triage, moderation, a guardrail) runs it from the real UI, and the answer DRIVES what happens next (the item lands in the chosen queue, the badge shows, the action is blocked) — not just printed.
• Both directions: a clearly-billing input routes to billing AND a clearly-technical one routes elsewhere. One label for every input is a broken integration.
• A low-confidence answer takes the app's fallback path (human review, "unsure" state) instead of being acted on.
• Provider errors (service down, bad key) show a visible, recoverable state — never a blank screen or an unhandled rejection.
• The call runs server-side: no provider request or key in the browser's Network tab.