NVIDIA Research · Efficient AI Team & Singapore Lab

Sol-AttnAccelerating Video Generation Inference via On-the-Fly Attention Sparsification

Haopeng Li, Yitong Li, Junsong Chen, Tian Ye, Haozhe Liu, Jincheng Yu, Duomin Wang, Ruihua Zhang, Zeke Xie, Enze Xie, Song Han

NVIDIA

Paper ↗Code ↗
Wan 2.1
Dense
2.02×
Sol-Attn
Hunyuan 1.5
Dense
2.12×
Sol-Attn
LTX 2.3
Dense
1.90×
Sol-Attn
Bernini
Dense
2.34×
Sol-Attn

Paper overview

Abstract

Training-free on-the-fly attention sparsification for efficient visual generation

Training-free dynamic sparse attention accelerates pretrained visual generators, but existing methods rely on costly offline routing and discard every unselected block. Sol-Attn unifies query-dependent threshold routing, sparse computation, and approximate correction within a single online-softmax pass. It selects blocks on chip without materializing a proxy map, then reuses below-threshold score columns to retain the long-tail contribution of skipped blocks.

Across image and video generation, Sol-Attn advances the quality-efficiency frontier of training-free sparse attention, delivering up to 2.1× speedup for video generation and 2.3× for video editing while preserving visual quality. Integrated with complementary acceleration techniques in Sol-Engine, it reaches up to 5× end-to-end speedup.

Dense vs. Sol-Attn

Visual comparisons.

Dense-attention and Sol-Attn outputs, aligned case by case.

01

LTX 2.3

1080p two-stage generation

1.90× end-to-end
INT. QUIET APARTMENT BY TALL WINDOW - LATE AFTERNOON. A bright cinematic 85mm close-up frames a beautiful woman in a cream sweater beside white curtains and wooden blinds. Her complete face fills the frame, with soft skin texture, clear eyes, and a calm natural expression; the background is a warm blurred room with plants and books, not futuristic. She starts looking down in partial shade. As wind moves the curtains, horizontal bands of golden sunlight travel slowly across her cheek, lips, and eyes. She raises her chin and turns from left profile into a clear three-quarter view, then gives a subtle thoughtful smile. The camera makes a slow push-in at eye level. Elegant natural portrait lighting, warm highlights, visible head motion, clean subject-background separation, no hands near face, no text, no logos. Audio: curtain fabric moving, distant city ambience, soft breath.Show full promptCollapse prompt

INT. QUIET APARTMENT BY TALL WINDOW - LATE AFTERNOON. A bright cinematic 85mm close-up frames a beautiful woman in a cream sweater beside white curtains and wooden blinds. Her complete face fills the frame, with soft skin texture, clear eyes, and a calm natural expression; the background is a warm blurred room with plants and books, not futuristic. She starts looking down in partial shade. As wind moves the curtains, horizontal bands of golden sunlight travel slowly across her cheek, lips, and eyes. She raises her chin and turns from left profile into a clear three-quarter view, then gives a subtle thoughtful smile. The camera makes a slow push-in at eye level. Elegant natural portrait lighting, warm highlights, visible head motion, clean subject-background separation, no hands near face, no text, no logos. Audio: curtain fabric moving, distant city ambience, soft breath.

INT. SMALL CINEMA PROJECTION ROOM - EVENING. A graceful female film archivist stands near an old projector in a tight 85mm portrait. Her full face is sharp and bright, with short wavy hair, natural pores, and a black turtleneck; dust particles sparkle in a cone of projector light behind her. She begins in side profile reading a film strip below frame. The projector starts, and a moving rectangle of warm flickering light sweeps across her face. She lifts her eyes and turns toward camera as the background alternates amber and cool shadow. The face remains dominant while film reels, shelves, and a round window blur behind her. Cinematic, tactile, elegant, strong light-on-face transition, no harsh cyberpunk styling, no text in focus, no hands in foreground. Audio: projector rattle, film reel clicking, room tone, one quiet inhale.Show full promptCollapse prompt

INT. SMALL CINEMA PROJECTION ROOM - EVENING. A graceful female film archivist stands near an old projector in a tight 85mm portrait. Her full face is sharp and bright, with short wavy hair, natural pores, and a black turtleneck; dust particles sparkle in a cone of projector light behind her. She begins in side profile reading a film strip below frame. The projector starts, and a moving rectangle of warm flickering light sweeps across her face. She lifts her eyes and turns toward camera as the background alternates amber and cool shadow. The face remains dominant while film reels, shelves, and a round window blur behind her. Cinematic, tactile, elegant, strong light-on-face transition, no harsh cyberpunk styling, no text in focus, no hands in foreground. Audio: projector rattle, film reel clicking, room tone, one quiet inhale.

INT. TAXI BACK SEAT - RAINY NIGHT, SOFT CITY LIGHTS. A close 85mm portrait observes a beautiful woman with shoulder-length hair by a rain-streaked taxi window. Her full face is bright and readable, with warm dashboard light on one side and cool street reflections on the other. She starts facing the window, eyes lowered, the city behind her reduced to creamy bokeh. As the taxi passes under a bridge, a sequence of amber, white, and blue light bands travels across her face. She turns her head toward camera and then looks past it, expression soft but alert. The camera remains intimate and stable with slight car motion. Realistic urban drama, face-first composition, strong light variation, clear head movement, no cyberpunk costume, no phone, no text. Audio: rain on glass, taxi turn signal, tires on wet road, quiet breath.Show full promptCollapse prompt

INT. TAXI BACK SEAT - RAINY NIGHT, SOFT CITY LIGHTS. A close 85mm portrait observes a beautiful woman with shoulder-length hair by a rain-streaked taxi window. Her full face is bright and readable, with warm dashboard light on one side and cool street reflections on the other. She starts facing the window, eyes lowered, the city behind her reduced to creamy bokeh. As the taxi passes under a bridge, a sequence of amber, white, and blue light bands travels across her face. She turns her head toward camera and then looks past it, expression soft but alert. The camera remains intimate and stable with slight car motion. Realistic urban drama, face-first composition, strong light variation, clear head movement, no cyberpunk costume, no phone, no text. Audio: rain on glass, taxi turn signal, tires on wet road, quiet breath.

Dense attention1.00×
Sol-Attn1.90×
02

Bernini

720p video editing

2.34× end-to-end
Moss coat
Turn the red raincoat into a living garment of emerald moss and small white flowers, and make the reflecting pool show a star-filled night sky, while preserving the woman, walk cycle, architecture, framing, and daylight.Show full promptCollapse prompt

Turn the red raincoat into a living garment of emerald moss and small white flowers, and make the reflecting pool show a star-filled night sky, while preserving the woman, walk cycle, architecture, framing, and daylight.

Glass fox
Turn the porcelain fox into transparent hand-blown glass containing a miniature thunderstorm with tiny clouds and lightning, while preserving its exact motion, silhouette, gallery, plinth, lighting, and camera path.Show full promptCollapse prompt

Turn the porcelain fox into transparent hand-blown glass containing a miniature thunderstorm with tiny clouds and lightning, while preserving its exact motion, silhouette, gallery, plinth, lighting, and camera path.

03

Wan 2.1

720p text-to-video

2.02× end-to-end

a motorcycle cruising along a coastal highway

a horse galloping across an open field

a train speeding down the tracks

Dense attention1.00×
Sol-Attn2.02×
04

Hunyuan 1.5

720p text-to-video

2.12× end-to-end

A cinematic tracking shot through a lantern-lit garden after rain, wet stone paths reflecting warm light, realistic camera movement, detailed plants and drifting mist.

A sweeping drone shot follows a red vintage sports car racing along a coastal cliff road at golden hour, waves crash below and sunlight flickers through sea mist.

A steam train moving on a mountainside

Dense attention1.00×
Sol-Attn2.12×

Outputs use identical prompts and seeds; speedups are end-to-end over the corresponding dense-attention pipeline.

Paper

Sol-Attn

Accelerating Video Generation Inference via On-the-Fly Attention Sparsification

Paper ↗GitHub ↗Back to top ↑
BibTeX
@misc{li2026solattnacceleratingvideogeneration,
      title={Sol-Attn: Accelerating Video Generation Inference via On-the-Fly Attention Sparsification},
      author={Haopeng Li and Yitong Li and Junsong Chen and Tian Ye and Haozhe Liu and Jincheng Yu and Duomin Wang and Ruihua Zhang and Zeke Xie and Enze Xie and Song Han},
      year={2026},
      eprint={2607.24027},
      archivePrefix={arXiv},
      primaryClass={cs.CV},
      url={https://arxiv.org/abs/2607.24027},
}