Auto-reframe
Crop and zoom the output so the subject is better framed — the “centre stage” behaviour from Meet, Zoom and Teams. Off by default.
new EffectsPipeline(stream, { reframe: true })It needs no extra assets and no extra model. It rides the face keypoints the network already produces for the matte, so turning it on costs a crop, not an inference.
Auto vs manual
Two modes, because apps ship both.
auto (the default) — the frame follows the subject. It holds still, makes a deliberate move when they’ve actually gone somewhere, then holds again. It is not a continuous tracker; continuous tracking reads as swimming, because a head moves constantly even when a person doesn’t.
new EffectsPipeline(stream, { reframe: true }) // auto, all defaultsmanual — solve the frame once, then freeze. Nothing moves until you ask. This is the mode for a “reframe” button in your own UI:
new EffectsPipeline(stream, { reframe: { auto: false } })
// later, from your button:
await pipeline.reframe()It reframes once when enabled, then holds. pipeline.reframe() recomputes and freezes again. In auto the camera already follows, so the method does nothing there.
Options
new EffectsPipeline(stream, {
reframe: {
zoom: 1.35,
gravity: 0.5,
margin: 0.04,
auto: { deadband: 0.09, ease: 0.07 },
},
})Anything you leave out takes its default.
How tight the crop is — 1 is the full frame, 1.35 shows ~74% of it. This is the requested zoom, not a guarantee: it relaxes toward 1 as needed to keep the subject framed (see below).
How hard the subject pulls the crop toward itself, 0–1. At 0 the frame never moves. At 1 the subject is centred exactly. The default deliberately sits in between — the crop moves toward you without centring you, which is what the commercial implementations do and what stops the effect feeling like a machine locking on.
Keep-out space around the subject’s face, as a fraction of the frame. Larger values give a roomier crop and make the no-op case (below) trigger sooner.
true (default) tracks; false is manual. Pass an object to tune the tracking — the two knobs only exist here because they describe how the frame moves, and in manual it doesn’t.
deadband (default 0.09) — how far the ideal frame must drift before the camera bothers moving. Bigger is lazier and steadier.
ease (default 0.07) — how quickly it closes the gap once it starts moving, per frame. Bigger is snappier; too big and it snaps.
Why it sometimes does nothing
If you sit in the far corner of frame, auto-reframe will do nothing at all. That’s correct, and it’s the same thing the commercial implementations do.
The crop has to stay inside the frame and contain the subject. For someone hard against an edge, containing them needs a crop about the size of the whole frame — which is to say, no reframe. There’s no special case for it; it falls out of the two constraints. As you move back toward the middle, the crop tightens again on its own.
It doesn’t move your background
The reframe applies to you, not to a virtual background:
new EffectsPipeline(stream, {
background: 'https://example.com/office.jpg',
reframe: true,
})The office stays put while you zoom — it’s a backdrop behind you, not part of the shot. A blurred background is different: that is your real room, so it reframes with you. Neither needs configuring.
Touch-up composes with reframe too; all three effects run together off the one encoder pass.
Which face
Auto-reframe frames one subject: the largest face, i.e. the person closest to the camera. The choice is sticky — a second person has to be decisively closer, not marginally, before the camera hands over. Two people sitting side by side won’t make it flip back and forth.
This is the opposite of touch-up, which smooths everyone. Framing is a choice about whose shot it is; smoothing isn’t.
Runtime control
await pipeline.setReframe(true) // enable, all defaults
await pipeline.setReframe({ zoom: 1.6 }) // enable, tighter
await pipeline.setReframe({ auto: false }) // manual mode
await pipeline.reframe() // manual: solve now
await pipeline.setReframe(null) // disableNo fetch, so enabling is instant.
Requirements
Auto-reframe needs face-capable model weights. Every current preset has them; if you’re self-hosting an older weight set the SDK logs a warning and leaves the effect off rather than failing your pipeline.