Longpipe

Open-source video effects SDK for the browser. Real-time virtual backgrounds and portrait matting on WebGPU with a WebGL2 fallback.

What it does

Wraps neural matting + compositing into a single class. Hand it a MediaStream, get a MediaStream back with the effect applied. Runs in a worker by default; audio passes through unchanged.

Quick links

• Getting started — install + your first effect
• API reference — EffectsPipeline options
• Architecture — the hydranet: one encoder, many heads
• Backgrounds — accepted background formats
• Touch-up — skin smoothing
• Auto-reframe — automatic subject framing
• Presets & autotune — model size knobs
• Self-hosting weights — point at your own CDN
• Datasets & licensing — MIT SDK, MIT weights, documented training data

Status

Early but functional — published on npm, with real-time matting, flow-based temporal stability (see Architecture) and audio denoising, benchmarking well against MediaPipe. Still planned: additional heads on the shared encoder — AR effects and lighting correction next. Expect rough edges. Issues + PRs welcome on GitHub.