Longpipe

Open-source video effects SDK for the browser. Real-time virtual backgrounds, face touch-up, auto-reframe and mic noise removal on WebGPU with a WebGL2 fallback.

What it does

Wraps neural matting + compositing into a single class. Hand it a MediaStream, get a MediaStream back with the effects applied. Runs in a worker by default; audio passes through unchanged unless you ask for denoising.

Backgrounds, touch-up and auto-reframe all ride a single shared encoder pass — enabling a second effect doesn’t cost a second inference. Audio denoise runs in a parallel AudioWorklet, off the GPU entirely.

Quick links

• Getting started — install + your first effect
• API reference — EffectsPipeline options
• Architecture — one encoder, many heads
• Backgrounds — accepted background formats
• Touch-up — skin smoothing
• Auto-reframe — automatic subject framing
• Audio denoise — real-time mic noise removal
• Presets & autotune — model size knobs
• Model layer — run the model without the pipeline
• Self-hosting weights — point at your own CDN

For LLMs and agents

The docs are published as plain markdown, no HTML parsing needed:

• /llms.txt — index of every page, with a summary of what Longpipe is
• /llms-full.txt — all documentation concatenated into one file

Both are generated from these same pages at build time, so they can’t drift.

Status

Early but functional, and under active development — published on npm, with real-time matting that benchmarks well against MediaPipe and frame-to-frame temporal stability from an optical-flow head. Backgrounds, touch-up, multi-face support, auto-reframe and audio denoise all ship today; AR effects and lighting correction are still planned. Expect API changes between versions, and rough edges. Issues + PRs welcome on GitHub.