GRAIL Documentation#
GRAIL — Generating Humanoid Loco-Manipulation from 3D Assets and Video Priors — is a fully digital data-generation pipeline for robot-compatible humanoid loco-manipulation. It composes 3D assets, simulator-ready scenes, robot-proportioned characters, and video foundation model priors to synthesize interactions in known metric 3D configurations, then reconstructs 4D human-object trajectories, retargets them to the Unitree G1, and trains task-general manipulation and locomotion trackers. The resulting egocentric visual policies transfer from GRAIL-generated data to real-world object pick-up and stair-climbing.
Tip
New here? Start with GRAIL Installation Guide, then Quick Start for a 5-minute smoke run.
News#
2026-06 — Released code on GitHub and dataset on HuggingFace.
2026-04 — Project page launched at research.nvidia.com/labs/dair/grail.
Pipeline at a glance#
Text / asset prompt
↓
python -m grail.pipelines.gen_terrain / python -m grail.pipelines.gen_3d_assets
→ 3D object meshes
↓
python -m grail.pipelines.gen_2dhoi
→ Blender + Kling AI HOI videos
↓
python -m grail.pipelines.recon_4dhoi
→ SMPL-X body + object 4D trajectories
↓
imports/GMR (retarget) → Unitree G1 joint trajectories
↓
imports/SONIC → Task-general tracking policy training
Sim-to-Real Deployment#
Representative real Unitree G1 deployment results are included for object pick-up and stair-climbing, alongside rendered egocentric views used for visual-policy training.
Rendered Egocentric Views#
Sim-to-Real Deployment#
Pick-up |
Stair-Climbing |
|---|---|
|
|
Bundled Dependencies#
Path |
Upstream |
Purpose |
|---|---|---|
|
GRAIL-vendored SONIC release tree |
RL training, inference, deploy stack |
|
SMPL-X → robot retargeting engine |
|
|
NVLabs GENMO / GEM-X |
Human pose estimation |
|
Object 6-DOF tracking |
|
|
Microsoft, Tencent |
Monocular depth + 3D asset generation |
Initialize with:
git submodule update --init --recursive
Getting Started
Pipelines
Data Exploration