TwelveLabs' Pegasus 1.6 reads egocentric video for robots

TwelveLabs, founded in 2021 and based in Seoul, has built a video intelligence platform around its Marengo and Pegasus models. With Pegasus 1.6, the company is extending that platform beyond enterprise video libraries into physical AI — a step it describes as its first model for egocentric, or first-person, footage. That footage is the type a worker would capture while cooking, assembling parts, or operating a robot remotely, and TwelveLabs argues it is easier to gather at scale than teleoperation data, which typically requires dedicated hardware and operators.

The release builds on Pegasus 1.5, whose Time-Based Metadata feature lets users define a custom schema and receive timestamped, structured data from video. Pegasus 1.6 adds improved entity recognition for tracking hands, objects, and tools across clips, plus faster, more cost-efficient processing for high-volume workloads. It arrives alongside existing collaborations with robotics labs and data teams, though the body does not name those customers or quantify the results they have seen.

The stakes hinge on whether robotics teams can turn ordinary first-person footage into training data that improves real machine behavior. TwelveLabs is explicit that it is not replacing a robot's tactile or actuator-level sensing; its role is to describe what happens in the video so teams can combine that with their own sensor data. Whether that combination speeds up robot training — and whether egocentric video proves as scalable as the company claims — is likely to determine how quickly the approach spreads beyond the labs already testing it.