English | 中文

XR Robotics is an open-source initiative for developers and professionals in XR, robotics, and AI. We provide XRoboToolkit, a suite of cross-device SDKs, frameworks, and sample applications that enable real-time remote robot teleoperation through XR devices.

• Website and paper: https://xr-robotics.github.io/
• YouTube channel: @XRRobotics
• PICO Developer samples: PICO Developer GitHub

📄 Each repository contains its own README and setup instructions.

Follow these steps to build and run a full XR-to-robot teleoperation sample on a PICO 4 Ultra headset and a Linux x86 PC. This sample has been tested only under the below system OS requirements: Linux x86 PC: Ubuntu 22.04 PICO 4 Ultra: User OS >5.12. Special permission with enterprise version and VST camera permission is required for headset camera access.

1. Install XRoboToolkit-PC-Service
   • Download deb package for ubuntu 22.04, or build from the repo source.
   • To install, use command

      sudo dpkg -i XRoboToolkit-PC-Service_1.0.0_ubuntu_22.04_amd64.deb

2. Clone & Set Up Python Teleop Sample
   • XRoboToolkit-Teleop-Sample-Python
3. Install the XR App on Headset
   • Turn on developer mode on Pico 4 Ultra headset first (Enable developer mode on Pico 4 Ultra), and make sure that adb is installed properly.
   • Download XRoboToolkit-PICO-1.1.1.apk on a PC with adb installed. ^{[Other Versions]}
   • To install apk on the headset, use command

      adb install -g XRoboToolkit-PICO-1.1.1.apk

4. Run Sample in Simulated Environment or on Real Robot
   • Connect robot PC and PICO 4 Ultra under the same network
   • On robot PC, double click app icon of XRoboToolkit-PC-Service or run service /opt/apps/roboticsservice/runService.sh
   • Open app XRoboToolkit on the PICO headset. Details of the Unity app can be found in the Unity source repo.
   • Follow instructions on XRoboToolkit-Teleop-Sample-Python to run test in simulated environment or on real robot.

• 🌍 Cross-device XR support via OpenXR
• 📡 Low-latency video streaming
  • <100ms latency, <1% packet loss for both video and command
• 🎮 Robot control via XR headset & controllers
• 🧪 Developer-friendly SDKs
  • Native C++ and Unity (URP/Android) clients
• 🧱 Modular, expandable architecture for future robot and headset support

• SEEC: Stable End-Effector Control with Model-Enhanced Residual Learning for Humanoid Loco-Manipulation
  from Georgia Institute of Technology and Tsinghua University

• TWIST2: A Scalable, Portable, and Holistic Humanoid Data Collection System
  from Amazon FAR, Stanford, USC, UC Berkeley, and CMU

• End-to-End Dexterous Arm-Hand VLA Policies via Shared Autonomy: VR Teleoperation Augmented by an Autonomous Hand VLA Policy for Efficient Data Collection
  from ByteDance Seed

• SONIC: Supersizing Motion Tracking for Natural Humanoid Whole-Body Control
  from Nvidia

This project is initiated and maintained by PICO as an open-source XR robotics framework for the community.

We welcome all contributors from academia, industry, and the open-source robotics community.

@article{zhao2025xrobotoolkit,
      title={XRoboToolkit: A Cross-Platform Framework for Robot Teleoperation}, 
      author={Zhigen Zhao and Liuchuan Yu and Ke Jing and Ning Yang}, 
      journal={arXiv preprint arXiv:2508.00097},
      year={2025}
}

• 💬 Join the XR Robotics Discord
• 🐞 Use Issues & Discussions in each repo
• 📩 Contact: Ning Yang (yangning726@gmail.com), Ke Jing (ke.jing@bytedance.com).

XRoboToolkit has been accepted to SII 2026 — The 2026 IEEE/SICE International Symposium on System Integration!

⚠️ This project is under active development. Contributions and feedback are highly encouraged.