@molecule/api-ai-image-generation-vllm-omni

vLLM-Omni image-generation provider for @molecule/api-ai-image-generation.

Generates and edits images on your own vLLM-Omni server through its OpenAI-compatible Images API (/v1/images/generations, /v1/images/edits). The default model is Qwen-Image-2.1 (text-to-image, multi-reference editing, transparent output, 2K native); the same bond serves any other diffusion model the server hosts — set VLLM_OMNI_MODEL or pass model.

Auto-generated, AI-first package reference for the molecule.dev ecosystem. It is written to be read by coding agents as much as by people, and is generated from this package's source — edit src/index.ts JSDoc, not this file.

Quick Start

import { requireProvider, setProvider } from '@molecule/api-ai-image-generation'
import { provider, transparentPrompt } from '@molecule/api-ai-image-generation-vllm-omni'

// Reads VLLM_OMNI_BASE_URL (e.g. http://localhost:8091) on each call.
setProvider(provider)

const { images } = await requireProvider().generate({
  prompt: transparentPrompt('A red sneaker, side view, studio lighting'),
  size: '2048x2048',
})
// images[0].base64 is PNG data — save it (uploads bond) before serving it.

Type

provider

Installation

npm install @molecule/api-ai-image-generation-vllm-omni @molecule/api-ai-image-generation @molecule/api-secrets

API

Interfaces

VllmOmniImageGenerationConfig

Configuration for the vLLM-Omni image generation provider. Every field falls back to its env var, read on each call (never at import time).

interface VllmOmniImageGenerationConfig {
  /** Server base URL without `/v1`. Defaults to the `VLLM_OMNI_BASE_URL` env var (required). */
  baseUrl?: string
  /** Optional API key, sent as `Authorization: Bearer …`. Defaults to `VLLM_OMNI_API_KEY`. */
  apiKey?: string
  /** Default model id. Defaults to `VLLM_OMNI_MODEL`, then `Qwen/Qwen-Image-2.1`. */
  defaultModel?: string
  /** Per-request timeout in milliseconds. Defaults to 300000 (diffusion at 2K is slow). */
  timeoutMs?: number
}

Classes

VllmOmniImageError

Error thrown when the vLLM-Omni server refuses or fails a request. Carries the server's HTTP status (0 when the server could not be reached or timed out) and its error code when it sent one. Deliberately NOT tagged with statusCode/errorKey, so API middleware answers its generic 500 instead of echoing the upstream status to the caller.

class VllmOmniImageError extends Error {
  /** HTTP status from the vLLM-Omni server; 0 for a network failure or timeout. */
  readonly status: number
  /** Error code or type from the server's error body, when present. */
  readonly code: string | undefined

  /**
   * Create the error.
   *
   * @param message - Human-readable message.
   * @param status - HTTP status (0 for network failure/timeout).
   * @param code - Server error code, when it sent one.
   * @param cause - The underlying error, if any.
   */
  constructor(message: string, status: number, code?: string, cause?: unknown) {
    super(message, cause === undefined ? undefined : { cause })
    this.name = 'VllmOmniImageError'
    this.status = status
    this.code = code
  }
}

Functions

createProvider(config)

Creates a vLLM-Omni image generation provider.

function createProvider(config?: VllmOmniImageGenerationConfig): AIImageGenerationProvider

• config — Optional overrides (base URL, API key, default model, timeout).

Returns: An AIImageGenerationProvider backed by a vLLM-Omni server.

isQwenImage21(model)

Whether a model id names Qwen-Image-2.1 (any org prefix or casing).

function isQwenImage21(model: string): boolean

• model — The model id sent to the server.

Returns: true for Qwen-Image-2.1.

nearestQwenImage21Size(size)

Maps a requested "WxH" size to the Qwen-Image-2.1 native size with the closest aspect ratio (e.g. 1024x1024 → 2048x2048, 1920x1080 → 2752x1536). Values that are not "WxH" (such as "auto") pass through.

function nearestQwenImage21Size(size: string): string

• size — The requested size.

Returns: A native Qwen-Image-2.1 size, or the input unchanged.

transparentPrompt(description)

Wraps a description in the prompt template that makes Qwen-Image-2.1 return a transparent (RGBA) image. Transparency has no request parameter — the prompt is the only switch.

function transparentPrompt(description: string): string

• description — What the image shows, e.g. "A red sneaker, side view".

Returns: The full prompt to send.

Constants

DEFAULT_TIMEOUT_MS

Default per-request timeout: 2K diffusion runs take tens of seconds or more.

const DEFAULT_TIMEOUT_MS: 300000

provider

The default provider, configured from env vars on each call (wire with setProvider). Safe to import before secrets are loaded.

const provider: AIImageGenerationProvider

QWEN_IMAGE_2_1_MODEL

The Qwen-Image-2.1 model id, as vLLM-Omni serves it. The bond's default model.

const QWEN_IMAGE_2_1_MODEL: 'Qwen/Qwen-Image-2.1'

QWEN_IMAGE_2_1_SIZES

Qwen-Image-2.1's native output sizes, keyed by aspect ratio (model card).

const QWEN_IMAGE_2_1_SIZES: Readonly<Record<string, string>>

QWEN_IMAGE_2_1_MAX_REFERENCE_IMAGES

Most reference images a single Qwen-Image-2.1 edit accepts.

const QWEN_IMAGE_2_1_MAX_REFERENCE_IMAGES: 10

aiImageGenerationVllmOmniSecretDefinitions

Secret definitions used by the vLLM-Omni image generation bond.

const aiImageGenerationVllmOmniSecretDefinitions: SecretDefinition[]

Core Interface

Implements @molecule/api-ai-image-generation interface.

Bond Wiring

Setup function to register this provider with the core interface:

import { setProvider } from '@molecule/api-ai-image-generation'
import { provider } from '@molecule/api-ai-image-generation-vllm-omni'

export function setupAiImageGenerationVllmOmni(): void {
  setProvider(provider)
}

Injection Notes

Requirements

Peer dependencies:

• @molecule/api-ai-image-generation >=1.1.0
• @molecule/api-secrets ^1.0.1

Environment Variables

• VLLM_OMNI_BASE_URL (required) — vLLM-Omni server URL
  • Setup: The base URL of your own vLLM-Omni server (started with vllm serve <model> --omni --port <port>), without the /v1 path.
  • Get it here: https://github.com/vllm-project/vllm-omni
  • Example: http://localhost:8091
• VLLM_OMNI_API_KEY (optional) — vLLM-Omni API key
  • Setup: Only if your vLLM-Omni server is started with an API key; sent as a Bearer token. Leave unset for a server without auth.
  • Get it here: https://github.com/vllm-project/vllm-omni
  • Example: my-server-key
• VLLM_OMNI_MODEL (optional) — vLLM-Omni model id
  • Setup: The model your vLLM-Omni server serves. Defaults to Qwen/Qwen-Image-2.1.
  • Get it here: https://github.com/vllm-project/vllm-omni
  • Example: Qwen/Qwen-Image-2.1

Runtime Dependencies

• @molecule/api-ai-image-generation

• @molecule/api-secrets

• Qwen-Image-2.1 WEIGHTS ARE RESEARCH-ONLY. They ship under the Qwen RESEARCH LICENSE AGREEMENT: "You shall not use the Materials for any commercial purpose without obtaining a separate commercial license" from Alibaba (model-business@notice.qwencloud.com). Research/evaluation use only — do NOT put it behind a production or paid feature, and do not offer it as a service, without that licence. vLLM-Omni itself is Apache-2.0; the licence follows the weights, whichever runtime serves them. Distributing the weights requires the Notice: "Qwen is licensed under the Qwen RESEARCH LICENSE AGREEMENT, Copyright (c) 2026 Hangzhou Tongyi Laboratory Technology Co., Ltd."

• You run the server; it needs a GPU. vllm serve Qwen/Qwen-Image-2.1 --omni --port 8091 (vLLM-Omni's own default port is 8000). A 7B DiT plus an 8B vision-language text encoder does not fit a molecule sandbox or the free tier. VLLM_OMNI_BASE_URL is required and has no default — a missing one throws the tagged config.notConfigured error. Give it WITHOUT /v1 (a trailing /v1 is stripped). VLLM_OMNI_API_KEY is optional and sent as Authorization: Bearer … only when set.

• Qwen-Image-2.1 sizes are 2K-native and requested sizes are snapped to the nearest aspect ratio: 1:1 2048x2048, 4:3 2400x1792, 3:4 1792x2400, 3:2 2528x1696, 2:3 1696x2528, 16:9 2752x1536, 9:16 1536x2752 (so 1024x1024 becomes 2048x2048). Other models get the size you send, unchanged.

• Results are base64 only (mimeType set; no URL). responseFormat: 'url', quality and style are not forwarded.

• Edits: the mask field is mask_image, not mask. This bond maps the core's mask for you (white = inpaint). Pointing the OpenAI bond at a vLLM-Omni server via OPENAI_BASE_URL sends mask, which the server drops — use this bond instead. Multi-reference edits: pass extra images in images (sent after image); Qwen-Image-2.1 takes at most 10 in total and more throws before any request.

• Guidance is off by default for Qwen-Image-2.1 (true_cfg_scale 1.0, 40 steps recommended). Do not assume SD-style cfg 4–7; negativePrompt likely has no effect unless guidanceScale > 1. In generateImage, guidanceScale → true_cfg_scale; in imageToImage it → guidance_scale (the edits API's field). strength is not forwarded.

• Diffusion parameters pass to the model WITHOUT server-side validation — a bad steps/seed/guidanceScale fails inside the model, not with a clean 400.

• Transparency has no parameter — it is triggered by the prompt template; use transparentPrompt(description).

• Errors are VllmOmniImageError with status (400 bad params, 422 missing fields, 503 engine not initialised; 0 = unreachable or timed out) and code when the server sent one. Nothing is retried; the default timeout is 300 s.

E2E Tests

Integration checklist — drive the real UI (live preview, no mocks), adapt each item to this app's actual screens/flows, and check every box off one by one. A box you can't check is an integration bug to fix — not a skip:

• Entering a prompt in the UI and submitting produces a REAL rendered image in the live preview — not a broken img (image icon / alt text), a grey placeholder, or an error toast. Confirm the img element actually decoded (its naturalWidth/naturalHeight are non-zero) and the picture visibly reflects the prompt (a "red bicycle" prompt shows a red bicycle).
• Two different prompts produce two visibly different images — a fixed stub, or a cached first result that never changes, is a broken integration.
• The image is STORED AND SERVED FROM THE APP'S OWN ORIGIN: the rendered img src (and any saved record) points at the app's uploads/storage, not the provider's temporary URL. Provider url results expire — reload the page (or revisit the record later) and the image must still load. Download the provider result server-side and persist it (uploads bond); never hotlink the provider URL — it 404s later and can leak the API request context. (base64/data results must likewise be saved, not held only in the response.)
• Any exposed generation options take effect: changing size/dimensions yields a differently-sized image; requesting a count (n) of N renders N images. If the UI exposes no such options, this box is n/a — say so.
• A rejected or policy-violating prompt, or a provider/rate-limit error, surfaces a clear message in the UI — not a crash, a blank screen, or a silently-broken img. The user can recover and try another prompt.
• Generation is server-side and authorized: the provider API key never reaches the browser (check the network tab / built client bundle — no key, no direct provider call from the page), the generate endpoint requires auth, and a caller cannot run unbounded costly generations through an open or unrate-limited route. Every image is billed.