A technical analysis argues that Chain of Thought reasoning—where AI models explicitly write out their reasoning steps—conflates readable output with actual computation and introduces unnecessary latency and cost. The field is shifting toward latent reasoning, where models perform internal reasoning without serializing it into tokens, using approaches like Coconut and HRM Text. This trade-off improves efficiency but deepens the challenge of understanding how models actually reason.
Summaries like this, in your inbox every morning.
Sign up free →What happened
A new analysis argues that Chain of Thought (CoT)—the technique where AI models show their reasoning step-by-step in text—is a useful shortcut but conflates readable output with actual computation. The piece identifies two concrete problems: CoT traces can diverge from what the model actually computed (producing plausible-sounding but wrong steps, or messy steps with correct answers), and generating longer traces inflates latency, cost, and token usage.
Why it matters
If CoT traces are unreliable audits of model reasoning, businesses and researchers relying on them for transparency or verification face a credibility gap. The observation that "generating text is not the same as thinking" suggests the current approach wastes compute on intermediate tokens that don't improve the final answer—a cost concern for anyone deploying reasoning models at scale.
What to watch
Recent work is shifting reasoning into latent space (internal numerical representations) rather than serialized text, with projects like Coconut, HRM / HRM Text, and RecursiveMAS moving reasoning out of the token stream. The article flags a trade-off: latent reasoning is more efficient, but opacity (the "black box wall") may make it harder to audit or verify how models actually reach conclusions.
The article presents a critique of Chain of Thought as a scaling trap—a technique that has become so standard it is often mistaken for a reliable window into model reasoning. The core tension is between readability and truthfulness: a step-by-step text trace appears to show the model's reasoning, but the model may produce a correct answer through an entirely different internal process, or generate plausible intermediate steps that lead nowhere. This faithfulness problem is compounded by a systems cost: every intermediate step must be tokenized and transmitted through the model's output, consuming memory and latency without necessarily improving the final answer.
The emergence of latent reasoning approaches (Coconut, HRM, RecursiveMAS) reflects a recognition that reasoning need not be serialized into human-readable text to be effective. By moving computation into internal representations and decoding only the final output, these methods sidestep the token-inflation penalty. However, the article hints at a new trade-off: latent reasoning sacrifices transparency for efficiency, introducing what it calls the "black box wall." This suggests that the field faces a deeper architectural choice: do we optimize for auditability (reasoning we can read and verify) or for efficiency (reasoning we cannot easily inspect)?
No comments yet. Be the first to share your thoughts!
Log in to join the discussion
Get curated AI news from 200+ sources delivered daily to your inbox. Free to use.
Get Started FreeFree · takes 30 seconds · unsubscribe anytime
1 minute a day. The AI essentials.
200+ sources · Email / LINE / Slack
