HiPHI dataset targets humanoid robot training gap

Humanoid robots need to learn balance, movement, and object interaction across many situations, but existing data sources have clear limits. Internet video offers diverse behavior yet cannot capture precise physical states, while laboratory motion capture records accurate movement but usually covers only a narrow set of actions. HiPHI aims to address both sides of that gap by combining optical motion capture at sub-millimeter accuracy with a wide range of actions.

The dataset's design leans on FrameNet, a linguistic framework for human action, to systematically organize whole-body motion coverage. It also pairs motion with synchronized object trajectories and meshes for 245.7 hours of human-object interaction, which the paper argues makes the data useful for teaching tasks such as carrying, pushing, and pulling. A benchmark suite for motion diversity and interaction grounding gives researchers a way to measure progress.

The white paper goes beyond data collection by reporting policies trained on the dataset and deployed on a physical Unitree G1 humanoid robot. Whether that sim-to-real transfer holds up across the full range of captured actions and objects, rather than the specific tasks tested, is likely to determine how broadly robotics teams adopt the dataset.