LongLive-Plug: Once-for-All Distillation for Video Generation

Shuai Yang^* Luozhou Wang^* Wei Huang ZhiFei Chen Bohan Zhang Xiao Fu Qianli Ma Chen-Hsuan Lin Weian Mao Bryan Chu Song Han Yukang Chen

NVIDIA

^* Equal contribution

LONG LIVE THE MODELS YOU BUILD.

Distill once. Unlock every task.

Developers build specialized video models on top of a shared backbone. Why should every downstream task need its own distillation?

Meet LongLive-Plug ↓

Task-specific distillation

Foundation models

Fine-tune for each task

···

Train every task

LongLive-Plug

Foundation models

Distill once

Few-step LoRACFG LoRALong-context LoRA

↓ Plug & play

···

No downstream training

02 / TRANSFER COVERAGE

A little plug.
A much wider world.

Diverse downstream tasks across backbone families.
One adapter set per backbone, reused across its family’s tasks.

Each pair preserves the source task’s settings. Expand a family to explore more downstream tasks. Open a card for its prompt, sampling settings and notes. Native schedules vary; MiniMax-H3 uses its published four-forward configuration.

03 / THE METHOD

One backbone.
Three reusable capabilities.

Distill guidance, sampling and long-context behavior into separate LoRAs. Compose the capabilities you need.

◈

Base video modelTrain a separate adapter set for each backbone

01Few-step LoRA

Fewer sampling steps.

02CFG LoRA

Guidance you can adjust.

03Long-context LoRA

Quality that lasts longer.

DISTILLED GUIDANCE

One adapter. Three strengths.

λ_cfg is the CFG LoRA weight. Guidance is distilled into one conditional forward per sampling step. The scales below show adapter weights, not the model’s native CFG setting.

CFG LoRA+Few-step LoRA→

50 → 4 steps12.5× fewer sampling steps

Compose the two adapters.
See the measured downstream outputs below.

04 / DOWNSTREAM TRANSFER

Same adapters.
New worlds to control.

Move the few-step and CFG LoRAs into a downstream model in the same family. Keep sampling fast; adjust guidance independently.

Downstream model+Few-step λ_step = 1+CFG λ_cfg adjustable→Fast & controllable

↳

No downstream retraining. These adapters are reused from the base model. All three results are shown together: only the distilled CFG LoRA weight changes, while the few-step weight stays fixed at 1.

05 / LONG CONTEXT

Keep the detail.
Long after the first frame.

For causal autoregressive models, transfer the long-context LoRA to reduce quality degradation over extended rollouts.

Selected late-rollout excerpts from the paper. Scores average seven VBench dimensions across all evaluation cases; this is not the official VBench overall score. Long-context transfer uses models that already support causal autoregressive generation.

06 / COMPARISON

Task-specific quality.
Without task-specific training.

Compare recorded outputs and benchmark results against per-task distillation. LongLive-Plug transfers directly from the backbone.

↳

Train once.
Let your models do more.

↑

07 / CITATION