COMPASS Handbook#

COMPASS is a cross-embodiment mobility policy framework that combines imitation learning, residual reinforcement learning, and policy distillation to ship one generalist navigation policy across humanoids, quadrupeds, and wheeled robots. This handbook is the live reference for installing, training, deploying, and extending it.
Residual RL Policy Distillation Isaac Lab 3.0 ROS2 deployment OSMO cloud VLA fine-tuning

↗ Project page · arXiv · Source on GitHub

End-to-end pipeline#

Each stage has a top-level entry-point script in the repo:

#

Stage

Script

1

Train residual RL specialist (or evaluate any policy)

run.py

2

Roll out specialists to collect HDF5 distillation data

record.py

3

Distil specialists into one generalist

distillation_train.py

4

Export to ONNX / JIT

onnx_conversion.py

5

Convert ONNX → TensorRT engine

trt_conversion.py

6

Deploy via ROS2 in Isaac Sim or on real robots

ros2_deployment/

Built on Isaac Lab 3.0 / Isaac Sim 4.5 and on top of NVIDIA’s X-Mobility base policy.

Where to start#

🚀 Quick start

Three commands to a training shell.

Quick start
📦 Installation

Docker-as-venv: three commands and you’re in a training shell.

Docker-as-venv install
🎓 Training a specialist

How the residual RL loop works, how to override embodiment / scene.

Training residual RL specialists
🛰️ ROS2 Deployment

ROS2 nodes consuming a TensorRT engine — Isaac Sim, sim2real, or object navigation.

ROS2 Deployment
☁️ OSMO cloud submission

Submit training and evaluation runs to NVIDIA’s OSMO cluster.

OSMO cloud submission
🤖 GR00T post-training

Use COMPASS distillation datasets to fine-tune NVIDIA’s Isaac-GR00T VLA model.

GR00T post-training (VLA fine-tuning)

What’s where in the repo#

COMPASS/
├── compass/
│   ├── residual_rl/         # Actor-critic that adds a residual on top of X-Mobility
│   ├── distillation/        # Generalist distillation (PyTorch Lightning trainer)
│   ├── rl_env/              # Isaac Lab `mobility_es` extension (embodiments + scenes)
│   └── utils/               # Shared helpers
├── configs/                 # Gin configs (train / eval / record / distillation / shared)
├── docker/                  # Docker-as-venv dev environment + Dockerfiles
├── osmo/                    # OSMO workflow YAMLs + Python launcher
├── ros2_deployment/         # ROS2 packages for Isaac Sim / real-robot deployment
├── scripts/                 # Standalone tools (HDF5→LeRobot, omap generator, …)
└── x_mobility/              # Vendored X-Mobility wheel

For deeper layout details, see CLAUDE.md.