VR Teleop Setup (PICO)

VR Teleop Setup (PICO)#

This page covers the one-time hardware and software setup for PICO VR whole-body teleoperation. After completing these steps, proceed to the ZMQ Manager tutorial to run teleop in sim or on real hardware.

Required Hardware#

Step 1: Install XRoboToolkit#

XRoboToolkit consists of a PC service (running on your workstation) and a PICO app (running on the headset) that streams body-tracking data.

PC Service#

  1. Go to XR-Robotics.

  2. Follow the “Install XRoboToolkit-PC-Service” instructions to install the PC service on your workstation.

  3. For onboard run sudo dpkg -i gear_sonic_deploy/thirdparty/roboticsservice_1.0.0.0_arm64.deb

PICO App#

  1. Wear the PICO headset to begin the setup and installation process.

  2. Complete the quick setup on PICO.

  3. Make sure the PICO is connected to Wi-Fi.

  4. Open the browser application in the PICO.

  5. Type “xrobotoolkit” in the search bar and select the GitHub page XR-Robotics.

../_images/google_search_screenshot.png

  1. Make sure Developer Mode is enabled (Settings → Developer).

  2. [INSIDE PICO] Scroll down in the GitHub page until you see the APK download option and click with the PICO trigger to download it.

Tip

Download XRoboToolkit-PICO-1.1.1.apk on PICO using the browser. (Other Versions)

  1. [INSIDE PICO] Open the manage downloads option on the top right section of the browser page and click to open the XRoboToolkit-PICO-1.1.1.apk download.

  2. [INSIDE PICO] Select Install — the application will appear in the Unknown section of your library.

Step 2: Motion Tracker Setup#

../_images/pico_setup_screenshot.png

  1. Strap one PICO motion tracker to your left ankle and one to your right ankle. Scrunch down any baggy clothing so the trackers are visible. Make sure the side with the light indicator faces up.

  2. Go to PICO settings. In the menu on the left, scroll down to the last option: “Developer”. Make sure “Safeguard” is turned off.

    • If the Developer option is not active, tap on “Software” until it appears.

  3. Click the Wi-Fi icon in the PICO menu. A picture of the headset will appear. Above the headset, there will be a small circular logo for the motion trackers. If there is no logo, open the “Motion Tracker” app itself.

    • Headset and 2 controllers will populate — select Motion Tracker (small circle).

  4. Next to each tracker, there is an “i” icon. Click on this and unpair all trackers.

  5. Once all trackers are cleared, click the “Pair” button in the top right corner.

  6. Press and hold the button on the top of each motion tracker for 6 seconds. Once in pairing mode, the lights will flash red and blue.

Motion Tracker Calibration#

  1. Wear the PICO headset over your eyes.

  2. Press the blue “Calibrate” button and follow the two calibration sequences:

    • Sequence 1: Stand stiff with the handheld controllers down by your sides.

    • Sequence 2: Look down at the foot motion trackers until the headset cameras recognize them.

  3. Once calibrated, wear the PICO headset around your forehead (ensuring PICO faces forward to continue detecting motion trackers).

Step 3: Install the PICO Teleop Environment#

From the repo root:

bash install_scripts/install_pico.sh

This creates a .venv_teleop virtual environment (Python 3.10) that includes:

  • teleop extra (ZMQ, Pinocchio, PyVista)

  • sim extra (MuJoCo, tyro)

  • XRoboToolkit SDK

  • Unitree SDK2 Python bindings

Activate it with:

source .venv_teleop/bin/activate   # prompt: (gear_sonic_teleop)

Step 4: Connect the PICO to Your Workstation#

Important

Before connecting the PICO, you must first launch the teleop streamer on your workstation so the PC Service has something to connect to:

source .venv_teleop/bin/activate
python gear_sonic/scripts/pico_manager_thread_server.py --manager \
    --vis_vr3pt --vis_smpl

Leave this running in a terminal throughout the setup steps below. This is part of the VR Whole-Body Teleop tutorial.

If the PICO cannot reach your workstation, check your firewall — disable it or add a rule to allow the XRoboToolkit port.

  1. Open the Wi-Fi settings on both the laptop/PC and PICO and ensure they are on the same Wi-Fi network. Take note of the Wi-Fi IPv4 address.

    • To find the PICO’s Wi-Fi, select the control center on the bottom right of the menu.

../_images/internet.png ../_images/pico_vr_screenshot.png

  1. Open the XRoboToolKit application. Enter the IP address of the laptop by clicking “Enter” next to “PC Service:”. You will know it is properly connected if WORKING appears next to “Status:”.

    • If your IP address is already inputted, select “Reconnect” where it says “Status:” in the Network section.

  2. Make sure the following boxes are ticked as shown in the picture below:

    • “Head” and “Controller” under the “Tracking” section.

    • For Data/Control, make sure to select the “Send” button.

    • For “Pico Motion Tracker” make sure to select “Full body”.

../_images/xrrobot_setup.png

Next Steps#

Your PICO hardware and software are now ready. Proceed to the ZMQ Manager (zmq_manager) tutorial to run whole-body teleoperation in simulation or on the real robot.

Contents