Live Volumetric Mapping MPCΒΆ
Fuse live RGB-D frames into a cuRobo volumetric map and MPC scene.
This reference example uses an Intel RealSense backend for live RGB-D capture:
it aligns depth to the RGB stream, converts each frame into
CameraObservation, and integrates the observations into
Mapper. The script is intended as a live mapping
and control demo: it updates the TSDF continuously, opens a Viser preview of
the reconstruction, recomputes an ESDF slice at the viewer refresh cadence,
and runs a Franka MPC loop with a draggable target against the live ESDF
collision grid. It does not write rendered PNGs or meshes.
The dynamic-obstacle clip shows the robot reacting to obstacles introduced into the live depth scene.
Install the RealSense Python bindings before running the example:
uv pip install pyrealsense2
Run live fixed-camera mapping until Ctrl-C. The example opens an
interactive Viser point-cloud preview while integrating:
python -m curobo.examples.reference.live_volumetric_mapping_mpc
The current RealSense backend provides RGB-D frames and intrinsics, but it does
not provide a camera-to-world pose. This example applies fixed -90 degree
X-axis and +90 degree Y-axis camera-frame rotations, then uses the first valid
depth frame to translate the camera so that the initial point cloud mean lands
in front of the robot base. Use --camera-pose x y z qw qx qy qz to set a
base transform before this fixed rotation and first-frame offset.