Long Video Generation | NVFP4 Infrastructure

LongLive-2.0: An NVFP4 Parallel Infrastructure for Long Video Generation

Yukang Chen^* Luozhou Wang^* Wei Huang^* Shuai Yang^* Bohan Zhang Yicheng Xiao Ruihang Chu Weian Mao

Qixin Hu Shaoteng Liu Yuyang Zhao Huizi Mao Ying-Cong Chen Enze Xie Xiaojuan Qi Song Han

NVIDIA

^*Equal Contribution

Paper

GitHub

Baseline versus NVFP4 key result overview

Play speed:

BF16

NVFP4

Why It Matters

Long-video training is expensive

AR long-video training needs to process long clean-history and noisy-target sequences, so GPU memory, VAE latent preparation, and DiT GEMMs all scale sharply with video length.

Deployment needs end-to-end speed

Real applications need more than diffusion-model FPS: KV cache, VAE decoding, and multi-shot continuous generation latency all have to be handled at the system level.

Demo Preview

Play speed:

BF16

NVFP4

Technical Walkthrough

Overall Framework

LongLive-2.0 treats algorithm and infrastructure as one system. On the training side, Balanced SP and NVFP4 make long-video AR fine-tuning practical. On the inference side, W4A4 execution, NVFP4 KV cache, parallel dequantization, and asynchronous VAE decoding improve end-to-end throughput.

Overview of the LongLive-2.0 training infrastructure

Balanced SP for AR Training

Each SP rank owns clean and noisy latents from the same temporal chunk. This matches the teacher-forcing attention mask, balances loss-bearing tokens across ranks, and lets VAE encoding be sharded by chunk instead of replicated on every GPU.

Clean Training Pipeline

Instead of relying on ODE initialization, short-video DMD, and extra long-tuning stages, LongLive-2.0 directly turns the base bidirectional diffusion model into a long, interactive, multi-shot AR model. Standalone LoRA weights then provide few-step inference capability.

NVFP4 Inference Infrastructure

At deployment time, LongLive-2.0 uses W4A4 inference, compresses the KV cache into NVFP4, and reconstructs cached chunks with a parallel dequantization kernel. Streaming VAE decoding overlaps video decoding with DiT denoising to reduce end-to-end latency.

NVFP4 DMD With LoRA

Few-step distillation keeps the generator and score models aligned with the W4A4 inference path. The quantized backbone stays fixed, while LoRA-only updates make the two-step generation recipe more stable and easier to transfer.

Multi-Shot Attention Sink

A fixed global sink preserves identity across the full video, while a shot-level sink is rebound at every scene change to maintain local temporal coherence. This supports minute-scale streaming without recomputing earlier history.

Key Results

2.1x 64s AR training speedup over BF16 + SP

45.7 FPS on GB200 with 2-step generation

19.4GB peak memory with NVFP4 KV cache

0.69x DMD peak-memory ratio with NVFP4 + LoRA

Setting	16s	32s	64s
BF16 baseline E2E generation	26.6s	53.2s	112.9s
LongLive-2.0 2-step E2E generation	11.2s	19.2s	36.3s

Training efficiency result — Training efficiency

Inference efficiency result — Inference efficiency

VBench short video result — VBench short-video quality

VBench long video result — VBench long-video quality

BibTeX

If you find this work useful, please consider citing: