@molecule/api-content-moderation-molecule

molecule.dev hosted content classifier for @molecule/api-content-moderation.

Scores text and images on molecule.dev and bills the check to your molecule project, so the app needs no OpenAI account. It is an ordinary bond: swap it for @molecule/api-content-moderation-openai (your own key) without changing code that calls the core.

Quick Start

import { setClassifier, requireClassifier } from '@molecule/api-content-moderation'
import { classifier } from '@molecule/api-content-moderation-molecule'

setClassifier(classifier) // reads MOLECULE_API_KEY from the environment

const verdict = await requireClassifier().check(comment.body)
if (verdict.flagged) {
  // Reject, or hold for review, BEFORE saving.
}

Type

provider

Installation

npm install @molecule/api-content-moderation-molecule @molecule/api-content-moderation @molecule/api-secrets

API

Interfaces

MoleculeModerationConfig

Options for createClassifier. Every field falls back to an env var, so the zero-argument classifier export works from .env alone.

interface MoleculeModerationConfig {
  /**
   * The molecule project API key (`mk_…`), or the sandbox token molecule.dev
   * writes for in-IDE apps (`mbk_…`). Defaults to `MOLECULE_API_KEY`.
   */
  apiKey?: string
  /**
   * The hosted services base URL, without a trailing slash. Defaults to
   * `MOLECULE_SERVICES_URL`, then `https://api.molecule.dev/api/v1/services`.
   */
  servicesUrl?: string
  /** Per-request timeout in milliseconds. Default 30000. */
  timeoutMs?: number
}

ProcessEnv

Environment variables this classifier reads.

interface ProcessEnv {
  MOLECULE_API_KEY?: string
  MOLECULE_SERVICES_URL?: string
}

Classes

MoleculeServiceError

Error thrown for a refused or failed hosted-service call.

class MoleculeServiceError extends Error {
  /** HTTP status from molecule.dev. */
  readonly status: number
  /** Stable error key from molecule.dev, e.g. `broker.error.budgetExceeded`. */
  readonly errorKey: string | undefined

  /**
   * Create the error.
   *
   * @param message - Human-readable message.
   * @param status - HTTP status.
   * @param errorKey - Stable error key, when the service sent one.
   * @param cause - The underlying error, if any.
   */
  constructor(message: string, status: number, errorKey?: string, cause?: unknown) {
    super(message, cause === undefined ? undefined : { cause })
    this.name = 'MoleculeServiceError'
    this.status = status
    this.errorKey = errorKey
  }
}

MoleculeContentClassifier

Classifier backed by molecule.dev's hosted moderation service.

class MoleculeContentClassifier implements ContentClassifierProvider {
  readonly name = 'molecule'

  private readonly apiKey: string
  private readonly servicesUrl: string
  private readonly timeoutMs: number

  /**
   * Create the classifier.
   *
   * @param config - Options; each falls back to its env var.
   */
  constructor(config: MoleculeModerationConfig = {}) {
    this.apiKey = config.apiKey ?? process.env.MOLECULE_API_KEY ?? ''
    this.servicesUrl = resolveServicesUrl(
      config.servicesUrl ?? process.env.MOLECULE_SERVICES_URL ?? DEFAULT_SERVICES_URL,
    )
    this.timeoutMs = config.timeoutMs ?? 30_000
  }

  /**
   * Classify text.
   *
   * @param content - The text.
   * @param options - Category filter and threshold (see the core's docs).
   * @returns Per-category scores and the overall decision.
   */
  async check(content: string, options?: ModerationOptions): Promise<ModerationResult> {
    return this.request('check', { content, ...pick(options) })
  }

  /**
   * Classify an image.
   *
   * @param image - The image bytes (at most 4 MB).
   * @param options - Category filter, threshold and MIME type (default `image/jpeg`).
   * @returns Per-category scores and the overall decision.
   */
  async checkImage(image: Uint8Array, options?: ImageModerationOptions): Promise<ModerationResult> {
    if (image.byteLength > MODERATION_SERVICE_LIMITS.maxImageBytes) {
      throw new MoleculeServiceError(
        `The image is ${image.byteLength} bytes. ${HINTS[413]}`,
        413,
        'hostedServices.error.inputTooLarge',
      )
    }
    return this.request('check-image', {
      image: Buffer.from(image).toString('base64'),
      mimeType: options?.mimeType ?? 'image/jpeg',
      ...pick(options),
    })
  }

  /** One POST to the hosted service. */
  private async request(
    operation: string,
    body: Record<string, unknown>,
  ): Promise<ModerationResult> {
    if (!this.apiKey) {
      throw new MoleculeServiceError(
        'MOLECULE_API_KEY is not set. Create a project API key on molecule.dev (or with `mlcl apikey create`) and put it in the API .env.',
        401,
        'hostedServices.error.tokenInvalid',
      )
    }
    const response = await fetch(`${this.servicesUrl}/moderation/${operation}`, {
      method: 'POST',
      headers: { authorization: `Bearer ${this.apiKey}`, 'content-type': 'application/json' },
      body: JSON.stringify(body),
      signal: AbortSignal.timeout(this.timeoutMs),
    })
    const text = await response.text()
    let data: unknown
    try {
      data = text ? JSON.parse(text) : {}
    } catch (error) {
      // A proxy/gateway error page, not the service: report the status only.
      throw new MoleculeServiceError(
        `molecule.dev returned ${response.status} with a non-JSON body.`,
        response.status,
        undefined,
        error,
      )
    }
    if (!response.ok) {
      const err = data as { error?: string; errorKey?: string }
      const hint = HINTS[response.status]
      throw new MoleculeServiceError(
        `molecule.dev moderation/${operation} failed (${response.status}): ${err.error ?? 'error'}${hint ? ` ${hint}` : ''}`,
        response.status,
        err.errorKey,
      )
    }
    return data as ModerationResult
  }
}

Functions

createClassifier(config)

Create a hosted content classifier.

function createClassifier(config?: MoleculeModerationConfig): ContentClassifierProvider

• config — Options; each falls back to its env var.

Returns: A ContentClassifierProvider backed by molecule.dev.

Constants

DEFAULT_SERVICES_URL

Default hosted services base URL.

const DEFAULT_SERVICES_URL: 'https://api.molecule.dev/api/v1/services'

MODERATION_SERVICE_LIMITS

Per-request limits of the hosted service.

const MODERATION_SERVICE_LIMITS: { readonly maxTextChars: 100000; readonly maxImageBytes: number }

classifier

The classifier implementation (wire with setClassifier).

const classifier: ContentClassifierProvider

contentModerationMoleculeSecretDefinitions

Secret definitions required by the molecule.dev hosted moderation bond.

const contentModerationMoleculeSecretDefinitions: SecretDefinition[]

Core Interface

Implements @molecule/api-content-moderation interface.

Injection Notes

Requirements

Peer dependencies:

• @molecule/api-content-moderation >=1.1.0
• @molecule/api-secrets ^1.0.1

Environment Variables

• MOLECULE_API_KEY (required) — molecule.dev project API key
  • Setup: Create one for your project with mlcl apikey create --project <id> --name <name> (keys start with mk_).
  • Get it here: https://www.molecule.dev
  • Example: mk_...

Runtime Dependencies

• @molecule/api-content-moderation

• @molecule/api-secrets

• Wire with setClassifier — NOT setProvider: this scores content only; it has no user reports or moderator queue.

• Config: MOLECULE_API_KEY (SERVER-side only) — a molecule project API key (mk_…) with scope broker or broker:moderation. Optional MOLECULE_SERVICES_URL (default https://api.molecule.dev/api/v1/services; required — plain-http is refused unless the host is loopback or a private-network endpoint (RFC 1918 / *.docker.internal, e.g. the sandbox gateway host.docker.internal)).

• Served by OpenAI's omni-moderation-latest, so category names are OpenAI's (harassment, hate/threatening, sexual/minors, violence/graphic, …), sorted highest score first. threshold re-decides each category by score; categories limits the verdict to a subset.

• At most 100,000 characters of text or a 4 MB image (png, jpeg, webp, gif) per check.

• Errors are MoleculeServiceError with status and errorKey (401 bad key, 402 allowance used up, 413 too large, 429 / 503 retry later). Nothing is retried. Decide per surface whether a failed check blocks the content or holds it for review — never publish silently on an exception.

E2E Tests

Integration checklist — drive the real UI (live preview, no mocks), adapt each item to this app's actual screens/flows, and check every box off one by one. A box you can't check is an integration bug to fix — not a skip:

• Every surface that accepts user-generated content (post, comment, image upload, bio — whatever this app has) runs it through check() / checkImage() SERVER-SIDE in the create/update handler, before persisting. Confirm the moderation call is on the write path, not client-side or skipped.
• Assert BOTH directions with real samples: clearly-violating content returns flagged: true and the UI rejects it with a visible reason; benign content returns flagged: false and publishes normally. A moderator that flags everything or nothing is broken.
• The decision gates persistence: a blocked item is NOT stored and is absent when you view the feed as a second user. Tighten ModerationOptions.threshold and a borderline item flips allowed → blocked.
• The failure mode is deliberate: when the moderation/AI call errors the request does NOT 500 — content is either published + logged (fail-open) or held for review (fail-closed) per the surface's intent.
• If the app has reporting, the round-trip works: a user's report() creates a 'pending' Report, it appears in the moderator queue via getReports(), and resolveReport() (approve/reject/dismiss) visibly changes its status and clears it from the pending queue.
• Moderator surfaces are privileged: a non-admin can't reach getReports() / resolveReport() (403) and can't see other users' flagged or pending content.