API reference

EffectsPipeline

The single public class. Hand it a MediaStream, get a MediaStream back with the effect applied. The pipeline owns its own worker, weight loading, autotune, frame transport selection, audio passthrough, and adaptive preset swaps — designed so you don’t have to wire any of that yourself.

Construction returns synchronously: pipeline.stream is wired immediately and emits the unprocessed input until the model is ready (~1–3s on cold start, depending on hardware). Use await pipeline.ready if you want to wait for the effect to actually be live before consuming the output.

const pipeline = new EffectsPipeline(inputStream, options?)

Constructor parameters

inputStream

Type: MediaStream  ·  Required

Any video stream — webcam from getUserMedia(), screen capture from getDisplayMedia(), or a captured stream from a <video> or <canvas> element via .captureStream(). Audio tracks are passed through unchanged by default; pass audio: 'drop' to strip them, or audio: 'denoise' to clean them (see Audio denoise).

options

Type: PipelineOptions  ·  Optional

Shape:

interface PipelineOptions {
  background?:        BackgroundInput              // default: 'blur'
  preset?:            PresetName | ManualPreset    // default: 'auto'
  weightsBaseUrl?:    string                       // default: cdn.longpipe.dev/.../v/0.0.4/
  audio?:             'passthrough' | 'drop' | 'denoise' | { denoise: DenoiseOptions }   // default: 'passthrough'
  enabled?:           boolean                      // default: true
  outputResolution?:  { w: number, h: number }     // default: input track size
  adaptive?:          boolean                      // default: true
  onReady?:           () => void
  onError?:           (err: PipelineError) => void
}

Each property in detail:

{
  background:  'blur'
}

{
  background: {
    blur: {
      strength?: number //0 to 1
    }
  }
}

{
  background: string | HTMLImageElement | ImageBitmap
}

{
  background: string | HTMLVideoElement
}

Properties

• pipeline.stream: MediaStream — the output stream. Available synchronously; emits passthrough video until the effect is live.
• pipeline.ready: Promise<void> — resolves when the worker emits its first composited frame. Optional to await.

Methods

• pipeline.setBackground(bg: BackgroundInput): Promise<void> — swap the background at runtime. Same input shape as the option. No flicker, no re-init.
• pipeline.setEnabled(enabled: boolean): void — toggle the effect. false puts the pipeline in passthrough mode.
• pipeline.setDenoise(input: boolean | DenoiseOptions): void — toggle or reconfigure audio denoise at runtime. false = passthrough; an object updates the DFN params live. Only effective when audio: 'denoise' was set at construction. See Audio denoise.
• pipeline.getAudioStats(): AudioStats | null — current denoise telemetry (model, per-hop p50/p95 ms, buffered latency, sample rate). null when denoise isn’t active.
• pipeline.setPreset(preset: PresetName | ManualPreset): void — manually swap presets. Disables the adaptive controller.
• pipeline.getStats(): Promise<PipelineStats> — runtime counters (model time, fps, current preset, etc). Single postMessage round-trip.
• pipeline.attachPreview(canvas: HTMLCanvasElement): void — attach a second canvas for previewing candidate effects. See Preview.
• pipeline.setPreview({ background, fps? }): Promise<void> — render a candidate effect to the preview canvas. See Preview.
• pipeline.clearPreview(): void — stop previewing.
• pipeline.destroy(): void — terminates the worker and releases GPU resources. The pipeline is not reusable after destroy.

Preview

Show a user what a candidate effect would look like — e.g. while they browse a background menu — without changing the effect on the outgoing stream. The main output keeps its currently-applied background; a second canvas renders the candidate. The network and matte are computed once and shared between the two, so previewing adds only a second composite (not a second inference).

// 1. Attach a canvas you own. Called ONCE — control is transferred to the
//    worker, so the element can't be drawn to by the page afterward. The
//    worker sizes the backing to the output resolution; CSS-scale it for display.
pipeline.attachPreview(document.querySelector('#preview'))

// 2. Preview a candidate. `background` is the identical wide surface as the
//    main effect (keyword, URL, ImageBitmap, { color }, { blur }, …). 'none'
//    previews the no-effect (raw) option. fps throttles the preview (default 15;
//    the main output stays the priority).
await pipeline.setPreview({ background: 'https://example.com/beach.jpg', fps: 15 })

// 3. Stop previewing (the canvas freezes on its last frame — hide it in your UI).
pipeline.clearPreview()

To apply a previewed candidate to the outgoing stream, just call pipeline.setBackground(sameInput) — there’s no separate “commit” call.

The preview canvas is local display only; it is not a MediaStream and is not transmitted. On WebGPU it composites directly to your canvas; on WebGL2 it’s presented via a per-frame snapshot of the render canvas (a little more expensive, which is why preview is throttled).

Error handling

onError and pipeline.ready rejection both fire for async errors after the constructor returns:

• Init failures — weight fetch 404, normalizeBackground URL fetch fail, worker init exception.
• GPU context loss — WebGL webglcontextlost event, WebGPU device.lost promise.
• Worker uncaught failures — pipe broken, command threw.
• Adaptive swap failures — marked recoverable: true on the error; pipeline keeps running on the prior preset.
• Runtime background errors — e.g., setBackground() called with an invalid URL.

onError does not fire for synchronous constructor errors — those propagate out of new EffectsPipeline(...) directly. Wrap in try/catch if you want a single handler for both.