Audio denoise

Real-time speech denoising — suppress background noise (keyboards, fans, traffic, room hum) from a microphone while keeping the speaker’s voice clean.

It runs as a separate pipeline from the video effect: an AudioWorklet on the audio render thread, in parallel with the GPU video work. The two share nothing but the output MediaStream, so denoise adds no load to the video pipeline.

Hear it

The same ~10-second clip — a noisy microphone recording, then cleaned by each model. Headphones recommended.

Noisy input

DFN3 — dfn (high)

DFN3 int8 — dfnint8 (mid)

RNNoise — rnnoise (low)

Enabling it

Pass audio: 'denoise'. The input stream’s microphone track is denoised in place; pipeline.stream carries the cleaned audio track.

const stream = await navigator.mediaDevices.getUserMedia({ video: true, audio: true })

const pipeline = new EffectsPipeline(stream, {
  background: 'blur',
  audio: 'denoise',
})

videoEl.srcObject = pipeline.stream   // denoised audio + processed video

pipeline.ready resolves on the first video frame and does not wait on audio — the mic passes through until the denoiser has loaded (a beat after init), then switches over automatically.

Models

Three models cover the hardware range. The default, 'auto', runs a tiny weight-free probe at init: it times the real network on the user’s device without downloading the multi-MB weights, then picks the best fit and downloads only that one.

The two DeepFilterNet models are Longpipe’s hand-written WebAssembly port of DeepFilterNet3; RNNoise is Xiph’s RNNoise.

Force a specific model (or tier shorthand) to skip the probe:

new EffectsPipeline(stream, { audio: { denoise: { model: 'dfn' } } })    // explicit model
new EffectsPipeline(stream, { audio: { denoise: { model: 'high' } } })   // tier: high | mid | low

Options

interface DenoiseOptions {
  model?:          'auto' | 'high' | 'mid' | 'low' | 'rnnoise' | 'dfn' | 'dfnint8'  // default 'auto'
  postFilterBeta?: number    // DFN post-filter — extra suppression of residual noise (default 0.03)
  gruLeak?:        number     // DFN recurrent-state drift bound (default 0.995)
  enabled?:        boolean    // start denoising vs. passthrough (default true)
}

postFilterBeta and gruLeak tune the DeepFilterNet models ('dfn' / 'dfnint8') only; RNNoise ignores them.

Runtime control

pipeline.setDenoise(false)                      // passthrough — stop denoising (cheap to re-enable)
pipeline.setDenoise(true)                       // resume
pipeline.setDenoise({ postFilterBeta: 0.05 })   // tune DFN params live

Inspect what’s running and how it performs:

const s = pipeline.getAudioStats()
// { model: 'dfn', p50Ms: 0.4, p95Ms: 0.6, latencyMs: 12.5, sampleRate: 48000, active: true }

p50Ms / p95Ms are per-hop compute time (the budget is one 2.67 ms audio render quantum); latencyMs is the buffering the denoiser adds.

Sample rate

The models run at 48 kHz. If the browser gives the AudioContext a different rate, the SDK resamples in and out automatically — you don’t need to do anything. getAudioStats().sampleRate reflects the actual context rate.

Notes

• Denoise needs an audio track in the input — request audio: true in getUserMedia. With no audio track it’s a no-op (a recoverable onError with source: 'audio').
• The DeepFilterNet models need wasm SIMD (every current browser has it); on a rare device without it, 'auto' falls back to RNNoise.
• This cleans your mic for outbound audio — in a call you send pipeline.stream to the other participant, whose playback is what they hear. If you monitor your own denoised audio locally, route it low-latency and use headphones (playing your live mic back is a feedback loop, the same as any mic monitor).