Long Video Generation | NVFP4 Infrastructure

LongLive-2.0: An NVFP4 Parallel Infrastructure for Long Video Generation

Yukang Chen^* Luozhou Wang^* Wei Huang^* Shuai Yang^* Bohan Zhang Yicheng Xiao Ruihang Chu Weian Mao

Qixin Hu Shaoteng Liu Yuyang Zhao Huizi Mao Ying-Cong Chen Enze Xie Xiaojuan Qi Song Han

NVIDIA

^*Equal Contribution

Paper

GitHub

Baseline versus NVFP4 key result overview

Play speed:

BF16

NVFP4

Why It Matters

Long-video training is expensive

AR long-video training needs to process long clean-history and noisy-target sequences, so GPU memory, VAE latent preparation, and DiT GEMMs all scale sharply with video length.

Deployment needs end-to-end speed

Real applications need more than diffusion-model FPS: KV cache, VAE decoding, and multi-shot continuous generation latency all have to be handled at the system level.

Demo Preview

Play speed:

BF16

NVFP4

Technical Walkthrough

Overall Framework

LongLive-2.0 treats algorithm and infrastructure as one system. On the training side, Balanced SP and NVFP4 make long-video AR fine-tuning practical. On the inference side, W4A4 execution, NVFP4 KV cache, parallel dequantization, and asynchronous VAE decoding improve end-to-end throughput.

Overview of the LongLive-2.0 training infrastructure

Balanced SP for AR Training

Each SP rank owns clean and noisy latents from the same temporal chunk. This matches the teacher-forcing attention mask, balances loss-bearing tokens across ranks, and lets VAE encoding be sharded by chunk instead of replicated on every GPU.

Clean Training Pipeline

Instead of relying on ODE initialization, short-video DMD, and extra long-tuning stages, LongLive-2.0 directly turns the base bidirectional diffusion model into a long, interactive, multi-shot AR model. Standalone LoRA weights then provide few-step inference capability.

NVFP4 Inference Infrastructure

At deployment time, LongLive-2.0 uses W4A4 inference, compresses the KV cache into NVFP4, and reconstructs cached chunks with a parallel dequantization kernel. Streaming VAE decoding overlaps video decoding with DiT denoising to reduce end-to-end latency.

NVFP4 DMD With LoRA

Few-step distillation keeps the generator and score models aligned with the W4A4 inference path. The quantized backbone stays fixed, while LoRA-only updates make the two-step generation recipe more stable and easier to transfer.

Multi-Shot Attention Sink

A fixed global sink preserves identity across the full video, while a shot-level sink is rebound at every scene change to maintain local temporal coherence. This supports minute-scale streaming without recomputing earlier history.

Key Results

2.1x 64s AR training speedup over BF16 + SP

45.7 FPS on GB200 with 2-step generation

19.4GB peak memory with NVFP4 KV cache

0.69x DMD peak-memory ratio with NVFP4 + LoRA

Setting	16s	32s	64s
BF16 baseline E2E generation	26.6s	53.2s	112.9s
LongLive-2.0 2-step E2E generation	11.2s	19.2s	36.3s

Training efficiency result — Training efficiency

Inference efficiency result — Inference efficiency

VBench short video result — VBench short-video quality

VBench long video result — VBench long-video quality

BibTeX

If you find this work useful, please consider citing:

@misc{chen2026longlive20nvfp4parallelinfrastructure,
      title={LongLive-2.0: An NVFP4 Parallel Infrastructure for Long Video Generation},
      author={Yukang Chen and Luozhou Wang and Wei Huang and Shuai Yang and Bohan Zhang and Yicheng Xiao and Ruihang Chu and Weian Mao and Qixin Hu and Shaoteng Liu and Yuyang Zhao and Huizi Mao and Ying-Cong Chen and Enze Xie and Xiaojuan Qi and Song Han},
      year={2026},
      eprint={2605.18739},
      archivePrefix={arXiv},
      primaryClass={cs.CV},
      url={https://arxiv.org/abs/2605.18739},
}