Kimodo-Generated Motion Preparation#

This guide covers converting motions generated by Kimodo (NVIDIA’s text-to-motion generation model) into ProtoMotions format for physics-based simulation.

Overview#

Kimodo can generate motions from text prompts in two skeleton formats:

  • SOMA skeleton (77 joints): output as .npz files at 30 fps

  • G1 skeleton: Unitree G1 robot, output as .csv files at 30 fps

Both formats can be converted to ProtoMotions .motion files and packaged into MotionLib .pt files for training.

Kimodo text-to-motion generation
     │
     ├── SOMA skeleton (.npz)  ──▶  convert_soma23_npz_to_proto.py
     │                                    │
     └── G1 skeleton (.csv)    ──▶  convert_g1_csv_to_proto.py
                                          │
                                          ▼
                                 ProtoMotions .motion files
                                          │
                                          ▼ (motion_lib.py)
                                 Packaged .pt MotionLib

SOMA Skeleton Motions (.npz)#

Kimodo generates SOMA-format motions as .npz files with the following fields:

  • local_rot_mats: (T, 77, 3, 3) — local rotation matrices for the full SOMA skeleton

  • posed_joints: (T, 77, 3) — global joint positions (root position at index 0)

  • foot_contacts: (T, 4) — foot contact labels (optional)

The converter subselects from 77 joints to the 23 actuated joints in the MJCF, dropping hands, finger details, face joints, and toe ends.

Note

An example Kimodo-generated SOMA .npz file is included at data/soma-kimodo-generated/ so you can try the pipeline without running Kimodo yourself.

Convert a directory of NPZ files:

python data/scripts/convert_soma23_npz_to_proto.py \
    --input-dir data/soma-kimodo-generated/ \
    --output-dir data/soma-kimodo-generated/proto \
    --input-fps 30 \
    --output-fps 30

Key arguments:

  • --input-dir: Directory containing .npz files (flat, no recursion)

  • --output-dir: Directory to save .motion files

  • --input-fps: Generation frame rate (default: 30)

  • --output-fps: Target output frame rate (default: 30)

  • --ignore-motion-filter: Skip quality filtering

  • --force-remake: Overwrite existing files

All converters also compute heuristic ground contact labels for each body (based on height and velocity thresholds). These are used by certain reward functions and observations during training (e.g., contact-aware tracking rewards).

Package and verify:

# Package
python protomotions/components/motion_lib.py \
    --motion-path data/soma-kimodo-generated/proto/ \
    --output-file data/soma-kimodo-generated/kimodo_soma_motions.pt

# Visualize
python examples/motion_libs_visualizer.py \
    --motion_files data/soma-kimodo-generated/kimodo_soma_motions.pt \
    --robot soma23 \
    --simulator isaacgym

G1 Skeleton Motions (.csv)#

Kimodo can also generate motions for the Unitree G1 skeleton as CSV files. These use a different convention from the BONES-SEED retargeted CSVs: positions are in meters, root orientation is a wxyz quaternion, joint angles are in radians, and there is no header row or frame index column.

Note

Example Kimodo-generated G1 CSVs and pre-converted .motion files are included at data/g1-kimodo-generated/ so you can try the pipeline. These are also the same motions on Kimodo front page showing real G1 deployment.

Convert a directory of CSV files:

python data/scripts/convert_g1_csv_to_proto.py \
    --input-dir data/g1-kimodo-generated/ \
    --output-dir data/g1-kimodo-generated/proto \
    --input-fps 30 \
    --output-fps 30 \
    --pos-units m \
    --rot-format quat_wxyz \
    --joint-units rad \
    --no-has-header \
    --no-has-frame-column \
    --force-remake

Package and verify:

# Package
python protomotions/components/motion_lib.py \
    --motion-path data/g1-kimodo-generated/proto/ \
    --output-file data/g1-kimodo-generated/kimodo_g1_motions.pt

# Visualize
python examples/motion_libs_visualizer.py \
    --motion_files data/g1-kimodo-generated/kimodo_g1_motions.pt \
    --robot g1 \
    --simulator isaacgym

Example: Text to Physics-Based Motion#

A typical end-to-end workflow using the G1 robot:

  1. Generate motions with Kimodo from text prompts

  2. Convert to ProtoMotions format

  3. Package into a MotionLib .pt file

  4. Train a physics-based policy to track the generated motions

  5. Deploy the policy to a real robot or test in simulation

# Step 1: Generate with Kimodo (see Kimodo docs)
# Output: kimodo_output/*.csv (G1 skeleton, 30 fps)
# Example CSVs are included at data/g1-kimodo-generated/

# Step 2: Convert (Kimodo G1 CSV format)
python data/scripts/convert_g1_csv_to_proto.py \
    --input-dir data/g1-kimodo-generated/ \
    --output-dir data/g1-kimodo-generated/proto \
    --input-fps 30 --output-fps 30 \
    --pos-units m --rot-format quat_wxyz --joint-units rad \
    --no-has-header --no-has-frame-column --force-remake

# Step 3: Package
python protomotions/components/motion_lib.py \
    --motion-path data/g1-kimodo-generated/proto/ \
    --output-file data/g1-kimodo-generated/kimodo_g1_motions.pt

# Step 4: Train
python protomotions/train_agent.py \
    --robot-name g1 \
    --simulator isaacgym \
    --experiment-path examples/experiments/mimic/mlp.py \
    --experiment-name kimodo_g1 \
    --motion-file data/g1-kimodo-generated/kimodo_g1_motions.pt \
    --num-envs 4096 \
    --batch-size 16384

# Step 5: Test in simulation / deploy to real robot
python protomotions/inference_agent.py \
    --checkpoint results/kimodo_g1/last.ckpt \
    --simulator isaacgym --num-envs 16

# For real robot deployment, see the G1 Deployment guide:
# https://protomotions.github.io/tutorials/workflows/g1_deployment.html

Supported Skeleton Formats#

Source

Format

Joints

Converter

ProtoMotions Robot

Kimodo SOMA

.npz

77

convert_soma23_npz_to_proto.py

soma23

Kimodo G1

.csv

23 DOFs

convert_g1_csv_to_proto.py

g1

BONES-SEED BVH

.bvh

77

convert_soma23_bvh_to_proto.py

soma23

BONES-SEED G1 CSV

.csv

23 DOFs

convert_g1_csv_to_proto.py

g1

Next Steps#