Installation

ProtoMotions supports five simulation backends: IsaacGym, IsaacLab, Genesis, Newton, and MuJoCo. You can install the simulation of your choice, and the simulation backend is selected via the configuration file.

Tested Versions:

Note

We recommend creating a separate virtual environment for each simulator to avoid dependency conflicts. We recommend using conda or venv for IsaacGym, Genesis, and MuJoCo, and uv for IsaacLab and Newton.

Prerequisites

After cloning the repository, fetch and check out files stored in Git LFS:

git lfs install
git lfs pull

This can take a while because pretrained checkpoints, motion files, meshes, and USD assets are large. If you fetch a subset of assets manually, make sure the files are checked out and not still Git LFS pointer files. Pointer files start with version https://git-lfs.github.com/spec/v1 and can cause errors such as is not a valid usda layer when IsaacLab loads robot assets.

Choose Your Simulator(s)

IsaacGym

IsaacGym requires Python 3.8.

1. Create a conda environment:

   conda create -n isaacgym python=3.8
   conda activate isaacgym
   

2. Download IsaacGym Preview 4:

   wget https://developer.nvidia.com/isaac-gym-preview-4
   tar -xvzf isaac-gym-preview-4
   

3. Install IsaacGym Python API:

   pip install -e isaacgym/python
   

4. Install ProtoMotions and dependencies:

   pip install -e /path/to/protomotions
   pip install -r /path/to/protomotions/requirements_isaacgym.txt
   

IsaacLab

We recommend using uv for IsaacLab installation. IsaacLab 2.x requires Python 3.11. For full installation details, see the IsaacLab Pip Installation Guide.

1. Create a virtual environment with uv:

   uv venv --python 3.11 env_isaaclab
   source env_isaaclab/bin/activate
   

2. Install PyTorch and IsaacLab:

   uv pip install torch==2.7.0 torchvision==0.22.0
   uv pip install isaaclab[isaacsim,all]==2.3.0 --extra-index-url https://pypi.nvidia.com
   

3. Install ProtoMotions and dependencies:

   uv pip install -e /path/to/protomotions
   uv pip install -r /path/to/protomotions/requirements_isaaclab.txt
   

Note

IsaacLab/IsaacSim may prompt for NVIDIA EULA acceptance on first use. Accept it interactively before running unattended headless jobs.

Genesis (Experimental)

Genesis requires Python 3.10.

1. Create a conda environment:

   conda create -n genesis python=3.10
   conda activate genesis
   

2. Install Genesis

3. Install ProtoMotions and dependencies:

   pip install -e /path/to/protomotions
   pip install -r /path/to/protomotions/requirements_genesis.txt
   

Newton

Newton is a GPU-accelerated physics simulator built on NVIDIA Warp, now available on PyPI. For full installation details, see the Newton Installation Guide.

Requirements: Python 3.10+ (3.11+ recommended), NVIDIA GPU (compute capability >= 5.0), driver 545+

1. Create a virtual environment:

   python -m venv .venv_newton
   source .venv_newton/bin/activate
   

2. Install PyTorch and Newton:

   pip install torch --index-url https://download.pytorch.org/whl/cu124
   pip install "newton[examples]"
   

   Use newton[sim] instead of newton[examples] if you only need headless mode (no viewer).

3. Install ProtoMotions and dependencies:

   pip install -e /path/to/protomotions
   pip install -r /path/to/protomotions/requirements_newton.txt
   

Note

On Python 3.10, imgui-bundle (a dependency of newton[examples]) has no prebuilt wheel and compiles from source, which can take 10-20 minutes. Python 3.11+ has prebuilt wheels and installs instantly.

MuJoCo (CPU-only)

MuJoCo is a CPU-only backend for quick testing and debugging without GPU. It supports single environment only (num_envs=1).

Requirements: Python 3.10+, No GPU required

1. Create a conda environment:

   conda create -n protomotions_mujoco python=3.10
   conda activate protomotions_mujoco
   

2. Install PyTorch CPU version (lighter, no CUDA needed):

   pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cpu
   

3. Install ProtoMotions and dependencies:

   pip install -e /path/to/protomotions
   pip install -r /path/to/protomotions/requirements_mujoco.txt
   

4. Run inference with MuJoCo:

   python protomotions/inference_agent.py \
     --checkpoint data/pretrained_models/motion_tracker/g1-bones-deploy/last.ckpt \
     --motion-file data/motion_for_trackers/g1_bones_seed_mini.pt \
     --simulator mujoco \
     --num-envs 1
   

   This example uses the shipped G1 motion tracker and matching motion data. The checkpoint directory includes the required resolved_configs_inference.pt file.

Note

MuJoCo backend is intended for quick policy validation and debugging. For training or large-scale evaluation, use GPU-accelerated backends (IsaacGym, IsaacLab, Newton, Genesis).

Troubleshooting

IsaacLab Issues

Torch Inductor Warning

On smaller GPUs, IsaacLab evaluation may print a warning similar to:

Not enough SMs to use max_autotune_gemm mode

This is a non-fatal PyTorch performance warning. Evaluation can continue unless it is followed by an actual traceback.

IsaacGym Issues

libpython Error

If you encounter libpython related errors, you need to set the LD_LIBRARY_PATH to your conda environment:

# First, check your conda environment path
conda info -e

# Then set LD_LIBRARY_PATH (replace with your actual conda env path)
export LD_LIBRARY_PATH=/path/to/conda/envs/your_env/lib:$LD_LIBRARY_PATH

# For example:
export LD_LIBRARY_PATH=${CONDA_PREFIX}/lib:$LD_LIBRARY_PATH

To make this permanent for only this conda environment, add activation hooks:

mkdir -p "${CONDA_PREFIX}/etc/conda/activate.d" "${CONDA_PREFIX}/etc/conda/deactivate.d"
cat > "${CONDA_PREFIX}/etc/conda/activate.d/isaacgym-libpython.sh" <<'EOF'
export _OLD_LD_LIBRARY_PATH="${LD_LIBRARY_PATH:-}"
export LD_LIBRARY_PATH="${CONDA_PREFIX}/lib:${LD_LIBRARY_PATH:-}"
EOF
cat > "${CONDA_PREFIX}/etc/conda/deactivate.d/isaacgym-libpython.sh" <<'EOF'
export LD_LIBRARY_PATH="${_OLD_LD_LIBRARY_PATH:-}"
unset _OLD_LD_LIBRARY_PATH
EOF

Memory Issues

If you run into memory issues during training:

# Reduce number of environments in your training command
--num-envs 1024

Next Steps

After installation, proceed to the Quick Start guide to train your first agent or run pre-trained models.