Production LLMs generate 54% more tokens on backend reasoning tasks not because they are less efficient, but because they must discover system state that backends don't provide cleanly.
When a team tested a full-stack RAG app, Supabase required 10.4M tokens and 10 manual fixes, while InsForge (a backend designed for agent operation) consumed 3.7M tokens and completed error-free.
The lesson: better models will try harder to fill information gaps, so the real cost reduction comes from backends that offer structured context and clear error codes upfront, not from larger models.
What happened
MCPMark V2 benchmarks showed that production LLMs achieve 54% higher token usage when reasoning through backend tasks, not because the models are less efficient but because they must discover system state that isn't handed to them cleanly. When an agent built a full-stack RAG app on Supabase, it consumed 10.4M tokens and required 10 manual interventions; the same app on InsForge (an agent-optimized open-source backend) consumed 3.7M tokens and completed without errors.
Why it matters
Most backends were designed for human developers using dashboards, so they return overly broad information (Supabase's OAuth docs include 5–10× more tokens than needed) and ambiguous error codes that force agents into retry loops. A smarter model tries harder to fill these gaps—triggering more discovery queries and reasoning—which is why token cost rises with a better Claude. The real fix is not a better model but a backend designed with agents in mind, offering structured context upfront and semantic exit codes.
What to watch
InsForge demonstrates the agent-first approach—a single CLI call returns the full backend topology in ~500 tokens before coding begins, with four narrowly scoped skills (CLI, debug, SDK, integrations) that activate only when needed. The GitHub repo has 14k+ stars. For teams building agent systems, this suggests that optimizing inference-time reasoning alone ignores the infrastructure gap: agents need backends that anticipate their information needs.
The article reveals a counterintuitive cost dynamic in inference-time reasoning: smarter models don't always mean cheaper inference. The culprit is architectural—most backend infrastructure (like Supabase) was built for human developers who can read dashboards and interpret errors, not for autonomous agents that must query for every piece of context. When a better LLM encounters missing information, it doesn't give up; it reasoning harder, runs more discovery calls, and enters retry loops when error codes are ambiguous. This behavior is rational from the model's perspective but expensive from the user's.
The token-cost spike from 3.7M to 10.4M in the same full-stack task demonstrates that the problem is not model intelligence but information design. InsForge's approach—returning the entire backend topology in a single ~500-token CLI call, using structured JSON, and assigning semantic exit codes—removes the discovery overhead. The agent's cognitive load stays lean because it only loads the skills it needs (CLI for table creation, debug for code failures, SDK for frontend, integrations for third-party auth), and it never faces ambiguous 401 errors that could mean three different things. This suggests a broader lesson: as inference-time reasoning becomes standard, teams will face diminishing returns from model scaling alone and must instead optimize the infrastructure agents operate on.
AI-summarized, only the topics you pick — one digest a day via Email, Slack, or Discord.
Free · takes 30 seconds · unsubscribe anytime
Ask AI anything about this article. Q&As are published on this page for other readers too.
A bundle of five online courses covering ChatGPT, AI agents, and Microsoft Azure AI Fundamentals certification…
T-Mobile US has outlined how it is integrating AI initiatives with customer outcomes, framing the effort as pa…
Charles Schwab and Zeplyn have integrated an agentic AI workflow into Schwab Advisor Center for wealth managem…
ServiceNow's President and CFO outlines a framework for making major AI investment decisions, emphasizing thre…
Researchers from Quantinuum, NVIDIA, and Pfizer validated a Generative Quantum AI (GenQAI) framework that comb…
KDDI's Tohoku division is offering a remote "AI Creator Class" for elementary school students that uses genera…
The AI news that matters, in one minute each morning.
Sign up free