Architecture

Longpipe’s video models are a hydranet: one shared encoder trunk feeding several small task heads. The design goal is that effect cost doesn’t stack — backgrounds, touch-up and auto-reframe all read from the same features, so enabling another effect adds a head, not another network.

Solid arrows are the full pass; dashed amber arrows are the skip-frame path.

The trunk

The encoder is an EfficientNet-Lite backbone (lite0 on most tiers, lite3 on xl), run at the preset’s base resolution with BatchNorm fused into the conv weights at export. It runs once per processed frame, and every head below reads its feature maps — this is the reason enabling touch-up or reframe doesn’t add an inference.

The next round of model R&D is replacing the backbone with EfficientFormerV2; the head interface stays the same, so it’s a weights swap rather than an architecture change for consumers.

The heads

• Matting head — a U-Net-style decoder producing the alpha matte, sharpened back up to canvas resolution by a lightweight refinement wrapper.
• Face keypoint head — 5-point heatmaps, decoded on the GPU into face boxes. This is what touch-up and auto-reframe ride; neither needs its own network.
• Optical-flow head — a motion field between the previous and current frame, used for temporal stability and for skipping encoder passes entirely (below).

Skipping frames without dropping them

The smaller tiers don’t run the trunk on every frame: small and medium run it every other frame, xs every third. On the frames in between, the flow head computes motion from the cached last-pass activations plus the previous and current frames — a far cheaper path than a full encoder pass — and that flow field warps the previous matte and face keypoints forward to the current frame. The compositor still renders every frame; skip frames just reuse warped outputs instead of fresh ones.

The same flow drives temporal stability on every tier: previous outputs are warped along the motion field and blended through a flow-gated stabilizer, which is why the matte holds steady frame-to-frame without any recurrent state in the network.

The landmark model

The 478-point face landmark model used by touch-up is deliberately not a head on the trunk. It’s a separate, ultralight network that runs only on the face crop the keypoint head found, and only when touch-up is enabled — so the trunk stays the same size whether or not you use it. On xs and small, landmark output is flow-warped between runs as well, the same trick used for the matte.

What’s next

Beyond the backbone swap, the roadmap adds more heads to the same trunk — AR effects and lighting correction are the next candidates — which is the payoff of the hydranet shape: each new effect is a head sharing the existing pass, not a new model.

Why the dragon theme? A hydra is a many-headed dragon, 龙 (Lóng) is Chinese for dragon and Longpipe is a many-headed network. 🐉