Molecule.dev

@molecule/api-ai-video-generation-kandinsky

Provider bond · ai-video-generation · API (Node) · v1.2.2 · Apache-2.0

Kandinsky 6.0 video generation provider for molecule.dev — self-hosted open-weight text/image-to-video-with-audio over a vLLM-Omni server's /v1/videos API

npm install @molecule/api-ai-video-generation-kandinsky

npm · Source on GitHub · Implements @molecule/api-ai-video-generation

How it works

@molecule/api-ai-video-generation-kandinsky is a provider bond on the API (Node) side: it implements the ai-video-generation core interface (@molecule/api-ai-video-generation) with a concrete vendor or library behind it.

Your code calls the core; you wire this provider once at startup. Swapping vendors later is one line in that wiring, not a rewrite.

import { requireProvider, setProvider } from '@molecule/api-ai-video-generation'
import { provider } from '@molecule/api-ai-video-generation-kandinsky'

// Reads KANDINSKY_BASE_URL (e.g. http://localhost:8091) on each call.
setProvider(provider)

const job = await requireProvider().generate({
  prompt: 'A golden retriever runs along a sunny beach, waves crashing',
  durationSeconds: 5,
})
let status = await requireProvider().getStatus(job.id)
while (status.status === 'pending' || status.status === 'processing') {
  await new Promise((resolve) => setTimeout(resolve, 5000))
  status = await requireProvider().getStatus(job.id)
}
// status.result carries the finished clip: `data` bytes (API-key servers)
// or a `url` — persist it before the server evicts the output.

Works with: @molecule/api-ai-video-generation, @molecule/api-secrets

Secrets: KANDINSKY_BASE_URL, KANDINSKY_API_KEY (optional), KANDINSKY_MODEL (optional)

Reference

Kandinsky 6.0 video-generation provider for @molecule/api-ai-video-generation.

Generates short video clips with a synchronized audio track on your own vLLM-Omni server (vllm serve kandinskylab/Kandinsky-6.0-… --omni) through its /v1/videos job API: submit (multipart), poll, then download the MP4 from /v1/videos/{id}/content. The default model is the 3B Lite distilled checkpoint; the same bond serves the Pro and base checkpoints the server hosts — set KANDINSKY_MODEL or pass model.

Auto-generated, AI-first package reference for the molecule.dev ecosystem. It is written to be read by coding agents as much as by people, and is generated from this package's source — edit src/index.ts JSDoc, not this file.

Quick Start

import { requireProvider, setProvider } from '@molecule/api-ai-video-generation'
import { provider } from '@molecule/api-ai-video-generation-kandinsky'

// Reads KANDINSKY_BASE_URL (e.g. http://localhost:8091) on each call.
setProvider(provider)

const job = await requireProvider().generate({
  prompt: 'A golden retriever runs along a sunny beach, waves crashing',
  durationSeconds: 5,
})
let status = await requireProvider().getStatus(job.id)
while (status.status === 'pending' || status.status === 'processing') {
  await new Promise((resolve) => setTimeout(resolve, 5000))
  status = await requireProvider().getStatus(job.id)
}
// status.result carries the finished clip: `data` bytes (API-key servers)
// or a `url` — persist it before the server evicts the output.

Type

provider

Installation

npm install @molecule/api-ai-video-generation-kandinsky @molecule/api-ai-video-generation @molecule/api-secrets

API

Interfaces

KandinskyVideoGenerationConfig

Configuration for the Kandinsky video generation provider. Every field falls back to its env var, read on each call (never at import time).

interface KandinskyVideoGenerationConfig {
  /** Server base URL without `/v1`. Defaults to the `KANDINSKY_BASE_URL` env var (required). */
  baseUrl?: string
  /** Optional API key, sent as `Authorization: Bearer …`. Defaults to `KANDINSKY_API_KEY`. */
  apiKey?: string
  /** Default model id. Defaults to `KANDINSKY_MODEL`, then the Lite distilled checkpoint. */
  defaultModel?: string
  /** Per-request timeout in milliseconds. Defaults to 300000 (a loaded GPU box responds slowly). */
  timeoutMs?: number
}

Classes

KandinskyVideoError

Error thrown when the vLLM-Omni server refuses or fails a request. Carries the server's HTTP status (0 when the server could not be reached or timed out) and its error code when it sent one. Deliberately NOT tagged with statusCode/errorKey, so API middleware answers its generic 500 instead of echoing the upstream status to the caller.

class KandinskyVideoError extends Error {
  /** HTTP status from the vLLM-Omni server; 0 for a network failure or timeout. */
  readonly status: number
  /** Error code from the server's error body, when present. */
  readonly code: string | undefined

  /**
   * Create the error.
   *
   * @param message - Human-readable message.
   * @param status - HTTP status (0 for network failure/timeout).
   * @param code - Server error code, when it sent one.
   * @param cause - The underlying error, if any.
   */
  constructor(message: string, status: number, code?: string, cause?: unknown) {
    super(message, cause === undefined ? undefined : { cause })
    this.name = 'KandinskyVideoError'
    this.status = status
    this.code = code
  }
}

Functions

isKandinskyDistilled(model)

Whether a model id names a distilled Kandinsky 6.0 checkpoint (any casing).

function isKandinskyDistilled(model: string): boolean
  • model — The model id sent to the server.

Returns: true for -distill checkpoints.

kandinskyDimension(value)

Snaps a pixel dimension to Kandinsky's 16-pixel grid (8× VAE compression times a 2× patch). Non-multiples are rejected by the pipeline, so 481 becomes 480 and 100 becomes 96; the grid has a FLOOR of 16 (a dimension below it would not describe a video), so 0 or a negative value snaps UP to 16. value must be finite — the provider refuses a non-finite width/height before reaching here.

function kandinskyDimension(value: number): number
  • value — The requested width or height in pixels (finite).

Returns: The nearest multiple of 16 that is at least 16.

framesForDuration(durationSeconds, fps)

Converts a clip length to Kandinsky's required 1 + 4k frame count (4k+1 frames; the 5-second clip shape is 121 frames at 24 fps).

function framesForDuration(durationSeconds: number, fps: number): number
  • durationSeconds — The requested clip length in seconds.
  • fps — The effective frame rate (defaults to 24).

Returns: A frame count of the form 1 + 4k, at least 1.

createProvider(config)

Creates a Kandinsky video generation provider.

function createProvider(config?: KandinskyVideoGenerationConfig): AIVideoGenerationProvider
  • config — Optional overrides (base URL, API key, default model, timeout).

Returns: An AIVideoGenerationProvider backed by a vLLM-Omni server.

Constants

KANDINSKY_6_LITE_DISTILL_MODEL

The default model id, as vLLM-Omni serves it: the 3B Lite distilled checkpoint — the only Kandinsky 6.0 variant that plausibly fits a single consumer/datacenter GPU at useful latency (10-step distill).

const KANDINSKY_6_LITE_DISTILL_MODEL: 'kandinskylab/Kandinsky-6.0-Lite-distill-5s-Diffusers'

KANDINSKY_6_DEFAULT_SIZE

The server's registered production geometry: width 864, height 480.

const KANDINSKY_6_DEFAULT_SIZE: '864x480'

KANDINSKY_6_FPS

Default frame rate of every Kandinsky 6.0 checkpoint (5 s = 121 frames).

const KANDINSKY_6_FPS: 24

KANDINSKY_6_DISTILL_STEPS

Inference steps a -distill checkpoint is trained for (model card: run distilled checkpoints at 10–16 steps).

const KANDINSKY_6_DISTILL_STEPS: 10

KANDINSKY_6_DISTILL_GUIDANCE_SCALE

Guidance scale every -distill checkpoint requires. The base checkpoints use 5.0, but distilled ones ship a PiflowScheduler that silently produces wrong results at any other value — the model card calls this out.

const KANDINSKY_6_DISTILL_GUIDANCE_SCALE: 1

KANDINSKY_6_DIMENSION_MULTIPLE

Kandinsky's VAE compresses 8× and patches 2× — every dimension must be divisible by 16.

const KANDINSKY_6_DIMENSION_MULTIPLE: 16

DEFAULT_TIMEOUT_MS

Default per-request timeout: a busy GPU box can be slow to answer even a submit.

const DEFAULT_TIMEOUT_MS: 300000

provider

The default provider, configured from env vars on each call (wire with setProvider). Safe to import before secrets are loaded.

const provider: AIVideoGenerationProvider

aiVideoGenerationKandinskySecretDefinitions

Secret definitions used by the Kandinsky video generation bond.

const aiVideoGenerationKandinskySecretDefinitions: SecretDefinition[]

Core Interface

Implements @molecule/api-ai-video-generation interface.

Bond Wiring

Setup function to register this provider with the core interface:

import { setProvider } from '@molecule/api-ai-video-generation'
import { provider } from '@molecule/api-ai-video-generation-kandinsky'

export function setupAiVideoGenerationKandinsky(): void {
  setProvider(provider)
}

Injection Notes

Requirements

Peer dependencies:

  • @molecule/api-ai-video-generation >=1.0.0
  • @molecule/api-secrets ^1.0.1

Environment Variables

  • KANDINSKY_BASE_URL (required) — Kandinsky server URL
    • Setup: The base URL of your own vLLM-Omni server serving Kandinsky 6.0 (started with vllm serve kandinskylab/Kandinsky-6.0-… --omni --port <port>), without the /v1 path.
    • Get it here: https://github.com/kandinskylab/kandinsky-6
    • Example: http://localhost:8091
  • KANDINSKY_API_KEY (optional) — Kandinsky API key
    • Setup: Only if your vLLM-Omni server is started with an API key; sent as a Bearer token. Leave unset for a server without auth.
    • Get it here: https://github.com/vllm-project/vllm-omni
    • Example: my-server-key
  • KANDINSKY_MODEL (optional) — Kandinsky model id
    • Setup: The checkpoint your vLLM-Omni server serves. Defaults to kandinskylab/Kandinsky-6.0-Lite-distill-5s-Diffusers.
    • Get it here: https://huggingface.co/kandinskylab
    • Example: kandinskylab/Kandinsky-6.0-Lite-distill-5s-Diffusers

Runtime Dependencies

  • @molecule/api-ai-video-generation

  • @molecule/api-secrets

  • You run the server; it needs a GPU. vllm serve kandinskylab/Kandinsky-6.0-Pro-5s-Diffusers --omni --host 127.0.0.1 --port 8091 --num-gpus 1 --enable-cpu-offload (the 29B Pro needs CPU offload even on 80 GB GPUs; the default 3B Lite-distill is the practical single-GPU checkpoint). KANDINSKY_BASE_URL is required and has no default — a missing one throws the tagged config.notConfigured error. Give it WITHOUT /v1 (a trailing /v1 is stripped). KANDINSKY_API_KEY is optional and sent as Authorization: Bearer … only when set. Generation is SLOW: the repo's benchmark table spans 356–3080 s per 5-s clip depending on GPU — budget poll loops accordingly.

  • Distilled checkpoints are pinned to guidance 1.0 + 10 steps here. Kandinsky -distill checkpoints ship a PiflowScheduler that produces silently wrong results at the base checkpoints' CFG 5.0 — the model card calls this out. When the resolved model id contains -distill and you set neither control, the bond sends guidance_scale=1.0 and num_inference_steps=10 itself; explicitly passing a guidanceScale other than 1 for a -distill model throws BEFORE any request. Base checkpoints pass your values through and default to the server's registered 5.0 / 50 steps.

  • Geometry is snapped, not rejected. Kandinsky's VAE forces dimensions divisible by 16 and frame counts of the form 4k+1: width/height are snapped to the nearest multiple of 16 (481→480) and durationSeconds becomes num_frames = 4k+1 at the effective fps (5 s at 24 fps → 121 frames — the model's native 5-s clip shape). Pass resolution ("864x480") to bypass snapping. With none of the three, the server's registered production geometry 864×480 applies. The built-in super-resolution cascade is NOT reachable through this port ("not supported" in the recipe), so upscale is absent — feature-detect as the core prescribes.

  • Audio is ON by default (unlike most vLLM-Omni models): Kandinsky samples a 44 kHz track whenever it generates. generateAudio: false sends sample_audio=false for video-only. cameraMotion is not forwarded (the Videos API has no camera control); negativePrompt forwards as negative_prompt.

  • Job ids are the server's own; statuses are normalized. queued → pending, in_progress → processing, completed/failed as-is; an unrecognized status word throws instead of polling forever. A completed job's result carries the video one of two ways: with KANDINSKY_API_KEY set, the content URL answers 401 to anyone without the key, so the bond downloads the MP4 from /v1/videos/{id}/content itself and returns the bytes as result.data; without auth, result.url points at that endpoint — fetch it before result.expiresAt (from the server's expires_at), or the server may evict the output. image becomes the input_reference multipart file part (first frame, image-to-video).

  • Kandinsky 6.0 code AND weights are MIT — no revenue cap or use restrictions, safe to expose to any app. (The LTX-2.5 weights this category's other bond calls are NOT: they carry the LTX-2.x Community License with a $10M revenue cap.)

  • Errors are KandinskyVideoError with status (400 bad params, 503 engine not initialised, 413 oversized upload; 0 = unreachable or timed out) and code when the server sent one. Nothing is retried; the default timeout is 300 s per HTTP call.

E2E Tests

Integration checklist — drive the real UI (live preview, no mocks), adapt each item to this app's actual screens/flows, and check every box off one by one. A box you can't check is an integration bug to fix — not a skip:

  • Entering a prompt in the UI and submitting produces a REAL playable video in the live preview once the job completes — not a broken video element, a spinner that never ends, or an error toast. Confirm the video element actually loaded (its readyState/duration are set) and the clip visibly reflects the prompt (a "waves on a beach" prompt shows waves, not a placeholder).
  • Two different prompts produce two visibly different clips — a fixed stub, or a cached first result that never changes, is a broken integration.
  • The video is STORED AND SERVED FROM THE APP'S OWN ORIGIN: the played source/src (and any saved record) points at the app's uploads/storage, not the provider's temporary URL. Provider url results expire — reload the page (or revisit the record later) and the video must still play. Download the provider result server-side and persist it (uploads bond) once the job completes; never hotlink the provider URL. (data results must likewise be saved, not held only in the response.)
  • Job progress is visible while polling: the UI shows the job is pending/processing (not frozen), and a failed job surfaces the provider's error message clearly so the user can recover and try another prompt.
  • Any exposed generation options take effect: changing durationSeconds/resolution yields a different-length/-sized clip. If the UI exposes no such options, this box is n/a — say so.
  • Generation is server-side and authorized: the provider API key never reaches the browser (check the network tab / built client bundle — no key, no direct provider call from the page), the generate endpoint requires auth, and a caller cannot run unbounded costly generations through an open or unrate-limited route. Every clip is billed or GPU-expensive.