Reka AI's Rho-1 omni-model handles text, images, video and robot control

Reka AI has a track record in multimodal AI: in April 2024 it shipped Reka Core, a multimodal language model that competed with GPT-4, Claude 3, and Gemini Ultra on benchmarks. The new Rho-1 release fits a broader push in AI research toward so-called world models, which aim to understand and simulate environments. Unlike most systems that route tasks to specialized models, Rho-1 runs all modalities as tokens in one shared context window with no tool calls or external models, and it generates continuous video in real time while responding to new instructions without restarting. The same weights that predict camera images also drive robot movements, with an inverse dynamics model extracting control signals from ordinary internet videos to work around scarce robot training data.

The outcome hinges on whether this unified approach can match or exceed specialized models in practice, and on how the research community responds to the trade-offs of a single model versus modular systems. For now, Rho-1 is a research preview, so its path to production and real-world deployment is likely still being explored.