TL;DRLong-WAM scales the visual history of a causal world-action model and keeps it real-time. Autoregressive video pretraining turns longer context into better control, and a co-designed runtime runs the full predict-then-act model at 107.4 ms per action chunk on an RTX 5090, with deployment on DGX Spark and Jetson AGX Thor.
Abstract
Real-time robot control demands enough visual history to infer motion and task progress, but processing that history can delay action. We present Long-WAM, a model–system framework for scaling the context of causal world-action models under real-time control constraints. Access to history does not by itself ensure effective context use: in our comparisons, longer histories amplify the advantage of AR video pretraining. We first learn causal prediction from robot and egocentric videos without action labels, then preserve this history-to-future structure during world-action adaptation. On RoboCasa GR-1, increasing context from 2.4 to 19.2 seconds raises success from 66.3% to 78.7%, while bidirectional initialization shows no net gain; robot-domain AR pretraining further improves peak success on GR-1 and LIBERO-Long. Long-WAM also achieves leading results on LIBERO-Long, RoboTwin 2.0, and DOMINO. Streaming observation encoding, asynchronous execution, and hardware-customized acceleration enable deployment on RTX 5090, DGX Spark, and Jetson AGX Thor while retaining future prediction. Inference takes 107.4 ms per action chunk on RTX 5090, including future-video latent prediction. Real-time deployment on Unitree G1 and YAM supports dynamic and long-horizon manipulation, including 95% success on dynamic cup stacking. As a memory-informed executor, Long-WAM also complements higher-level planning in composite tasks.
01 · Method
Inverse-dynamics (IDM) inference: the video expert predicts future latents from the observed history; the action expert denoises an action chunk conditioned on that history and the predicted future. Video queries never read action tokens.
LongLive2.0-Robot continues LongLive-2.0’s AR checkpoint on ≈10,000 window-equivalent hours of robot and egocentric video (RoVid-X, AgiBot World, EgoDex, EgoVerse, VITRA). It learns teacher-forced next-chunk prediction on sequences up to 30 s, without action labels.
Video and action experts are coupled through an asymmetric attention interface (a Mixture of Transformers). Video queries never read action tokens. Action queries read observed history, partially denoised future latents and the action chunk, so Long-WAM predicts the future first and then acts, without decoding pixels.
We scale the duration of real observations in the causal prefix and keep the forecast and action horizons fixed. Each window is trained separately and evaluated at its own length, up to 38.4 s on RoboCasa GR-1.
Pure asynchronous execution, streaming VAE encoding, NVFP4 quantization, KV reuse, CUDA Graph and device-specific kernel tuning bring the full model to RTX 5090, DGX Spark and Jetson AGX Thor.
02 · Context scaling
Both autoregressive initializations turn longer history into higher success. The bidirectional initialization shows no net gain on RoboCasa GR-1 between 2.4 s and 19.2 s. On LIBERO-Long, 2.4 s of history lifts success from 94.5% to 99.5%.
Context windows are equally spaced on the x-axis. At 38.4 s, 80.4% of sampled history frames are padding (training trajectories average 12.1 s). The drop there reflects limited effective history, not an established memory limit.
03 · Simulation benchmarks
Long-WAM (IDM) reaches 99.5% average on LIBERO and 94.4% on RoboTwin 2.0. After dynamic-data fine-tuning it reaches 34.9% success on DOMINO, the highest among the compared methods.
04 · Real world
Unitree G1 grasps from a moving conveyor at up to 7.5 cm/s; both baselines fail every trial at 6.0 and 7.5 cm/s. Dynamic cup stacking succeeds in 19 of 20 trials versus 0 of 20 for both baselines. On YAM, tasks lasting over 40 s average 81.7% success.
05 · Real time on the edge
Shared optimizations and device-specific kernel tuning speed up the full video-then-action pipeline by 3.3×, 4.1× and 3.2× over BF16 eager on RTX 5090, DGX Spark and Jetson AGX Thor.
Long-WAM (IDM) · V4/A4 · full observation VAE included · each stage keeps the previous optimizations.
Long-WAM keeps 94.2% success asynchronously (94.4% synchronously) with the lowest overlap RMSE and jerk. The AR video foundation conditions actions on temporally continuous predictions.
06 · GPT-6 Astra + Long-WAM
Coupled with GPT-6 Astra, the unchanged Long-WAM checkpoint rises from 31.4% to 54.4% overall on RoboCasa365. On unseen compositions it rises from 6.1% to 35.0%.
07 · Demos
Every clip is labelled with its source and playback speed.
Each clip is generated from a single image and a language instruction. They are video predictions, not executed rollouts.
Left: Long-WAM alone. Right: GPT-6 Astra plans, handles recovery and corrects actions when needed, while Long-WAM executes. Both sides use the same checkpoint. The SUCCESS and FAILED badges come from the original recordings.
Citation
@article{huang2026longwam,
title = {Long-WAM: Scaling the Context of World-Action Models},
author = {Huang, Wei and Zhang, Bohan and Liu, Chenzhi and Liu, Isabella and
Yang, Shuai and Mao, Weian and Wang, Luozhou and Xiao, Yicheng and
Lin, Weifeng and Hu, Qixin and Chu, Bryan and Liu, Sifei and
Fan, Linxi and Qi, Xiaojuan and Han, Song and Chen, Yukang},
journal = {arXiv preprint},
year = {2026}
}