AI companies have built the technology and promised transformation, but the path between those two remains unclear, creating a vacuum filled by unverified claims.

A February study by Mercor, an AI hiring startup, tested AI agents powered by models from OpenAI, Anthropic, and Google DeepMind on 480 workplace tasks performed by bankers, consultants, and lawyers. Every agent tested failed to complete most of its duties.

An Anthropic study predicted which jobs would be most affected by LLMs (large language models—AI systems that understand and generate text), but the researchers acknowledge their predictions are guesses based on what tasks LLMs seem good at rather than how they actually perform in real workplaces.

The article identifies a gap between AI companies' claims of transformation and evidence of real-world deployment: workflows are contaminated with people and existing systems, and adding AI sometimes makes things worse. Closing this gap requires transparency from model makers, coordination between researchers and businesses, and new evaluation methods for real-world rollout.