Google DeepMind shows video generators solve computer vision tasks with minimal training data

Google DeepMind researchers unveiled GenCeption, a model that repurposes a pre-trained video generation system for classical computer vision tasks including depth estimation, surface normal prediction, segmentation, and 3D pose recognition. The core innovation is architectural simplification: while diffusion models typically generate video through many small iterative steps starting from noise, GenCeption produces all outputs in a single forward pass, dramatically improving speed while preserving accuracy.

The model builds on Alibaba's open-source Wan2.1 video generator, adapting it to accept task instructions via text prompt—similar to how chatbots respond to natural language commands. Rather than creating separate specialized architectures for each vision task, the team represents all outputs as standard three-channel RGB images: depth maps and surface normals are encoded as color images, segmentation masks as image-like representations, and camera motion as an image. For outputs that don't naturally map to images, such as 3D keypoint coordinates, trainable modules are added. A single loss function trains across all tasks simultaneously, with task-specific requirements handled by preprocessing the training data rather than redesigning the network.

The training data was almost entirely synthetic, consisting of just 7,500 videos. The team synthesized these by combining 800 digital human models with 200 motion capture sequences, then rendered the results in Blender with varied backgrounds and camera angles; only language-guided segmentation drew from real video. Despite this minimal and artificial training set, GenCeption matched or exceeded state-of-the-art results across multiple benchmarks. Its depth estimates match DepthAnything 3, it outperformed NormalCrafter and Lotus-2 on surface normal estimation, beat Genmo and TRAM on 3D pose recognition, and matched Meta's SAM 3 combined with Gemini 3.5 Flash on complex language-guided segmentation. Other models such as D4RT and VGGT Omega trained on millions of videos; GenCeption achieved comparable results using 7 to 500 times less data. When tested under identical conditions, pretraining on video generation also outperformed methods such as V-JEPA and VideoMAE V2, suggesting the generation task itself—not data volume alone—teaches useful spatial representations.

Generalization beyond the training distribution was particularly striking. GenCeption trained almost exclusively on synthetic videos of single people but successfully processed real footage with multiple people, animals, and humanoid robots—categories it never encountered during training. In some cases, the model's outputs contained more detail than the Blender renderings used for training, preserving fine features like a cat's whiskers and individual strands of hair. However, joint training across all tasks did degrade performance on 3D keypoint estimation, likely because the additional modules needed for that task interfered with mechanisms the base model learned during video pretraining; the researchers conclude that minimal architectural changes to the original model yield the best results. Processing speed remains a constraint: the smaller model requires approximately six seconds to process an 81-frame video, while the larger 14-billion-parameter variant takes about ten seconds.

The authors argue that these results demonstrate video generators already contain a type of universal "world model"—an internal representation of space, motion, and object geometry—that could serve as the foundation for computer vision in the same way large language models serve text. This claim is contested. An international research team recently established a formal definition of world models (OpenWorldLib) and explicitly excluded text-to-video generators, citing their lack of grounding in real-world feedback. Yann LeCun, former Meta chief AI scientist, has gone further, claiming generative video models are a dead end; Meta's alternative approach, V-JEPA 2, predicts abstract concepts rather than pixels. A benchmark from Tsinghua University documented fundamental limitations: Sora 2, Seedance 2.0, and Veo 3.1 repeatedly failed basic physics and logic tests even when their visual output appeared convincing. GenCeption's contribution is narrower but more pragmatic: the researchers extract learned features from video generation for specific, well-defined tasks (depth, segmentation, pose) where those representations outperform specialized systems. Google DeepMind is exploring complementary directions with Genie 3, which builds interactive 3D environments for training AI agents.

Computer vision has historically fragmented into specialized models—separate architectures for segmentation, depth estimation, surface normals, and pose recognition—each optimized for its narrow task. The field has lacked the kind of unified training signal that made large language models so versatile; learning to predict the next word proved to be a powerful general-purpose objective that forced models to absorb grammar, world knowledge, and relationships. GenCeption suggests that video generation may serve a similar role for vision. Generating realistic video requires models to learn spatial geometry, object movement, and basic physics—representations that turn out to be useful far beyond video synthesis. By building on Alibaba's open-source Wan2.1 model and simplifying its architecture (replacing the iterative diffusion process with a single forward pass), the researchers demonstrate that these learned features can outperform task-specific systems trained on vastly more labeled data. The use of text prompts to specify which task to perform, combined with a unified output representation (all results expressed as standard three-channel RGB images), further echoes the instruction-following paradigm of large language models.

However, the claim that video generators contain a true "world model" remains contested. An international research team recently proposed a formal definition of world models and explicitly excluded text-to-video models from consideration, citing their lack of feedback from the real world. Yann LeCun, former Meta chief AI scientist, has argued that pixel-prediction video models are fundamentally limited, and a Tsinghua University benchmark showed that models like Sora 2, Seedance 2.0, and Veo 3.1 repeatedly failed basic physics and logic tests despite visually convincing output. GenCeption sidesteps this debate by using video generation for a narrower purpose—extracting learned features for specific tasks—rather than asking the model to predict how the world will behave. The results show that even without explicit physics understanding, the representations learned during video generation contain enough spatial and motion information to solve dense prediction tasks more efficiently than traditional approaches.