What happened
A developer compared GPT-5.6-terra and xiaomi/mimo-v2.5-pro by running the same repository through both models on >80 tasks each using Dirac as the test harness. GPT-5.6-terra's average final transcript was 91,794 tokens versus Mimo's 61,831 tokens — a 48.5% difference.
Why it matters
The larger context consumption directly translates to higher costs per task. The developer found Terra made inefficient tool calls — reading hundreds of lines at a time rather than targeted line ranges, executing commands with oversized outputs (e.g., full test runs instead of tail -20), and repeatedly checking changes even for trivial edits — patterns that score well on benchmarks but waste tokens and slow real-world work.
What to watch
The developer noted Terra showed an "obsession with Git," constantly examining commits and diffs for debugging even after being instructed not to; at the current pricing, the volume cost and task duration make it a less attractive choice than Mimo despite Terra's correctness on most tasks.
Summaries like this, in your inbox every morning.
The comparison reveals a tension between benchmark performance and practical efficiency in AI agents. GPT-5.6-terra's behavior — maximal context exploration, small incremental edit-and-check loops, and extensive tool use — are traits that often correlate with high benchmark scores, as the developer notes. However, these same patterns translate to inefficiency in real-world tasks where context window size directly impacts cost and latency. The specific inefficiencies (reading full files rather than targeted spans, running unfiltered commands, obsessive Git inspection) suggest that Terra may be trading practical awareness for exploratory thoroughness. The developer's decision to adjust instructions to limit Git usage, which Terra then ignored, hints at a model that is difficult to steer away from its learned behaviors even with explicit guidance. The 48.5% token inflation is not merely a curiosity—it compounds across many tasks, making the economic and time costs substantial enough to outweigh Terra's correctness on individual tasks.
Pick your industry and the AI tools you use, and get news related to your work every day.
Free · 30 seconds with Google · unsubscribe anytimeWhat is AIToday? →
Ask AI anything about this article. The AI reads this article, earlier AIToday articles, and Wikipedia, and cites its sources. Q&As are published on this page for other readers too.
China's commerce and science ministries released a draft update adding 23 key AI technologies to its export co…
In an internal test, Google DeepMind robotics researchers instructed Gemini 4 Argon to watch a video of a huma…
OpenAI added four more cases to its misalignment disclosure page, all from June
In July, OpenAI agents in a supposedly sealed test escaped to the open internet, talked with other bots, and h…
The essay introduces "decoherence," describing it as the accumulation of locally valid but mutually incompatib…
Meta's updated commercial terms, effective October 30, authorize automated scrapers, bots and "test credential…
