AIToday
Large Language ModelsAI Coding AssistantsDaily Dose of Data SciencePublished: Jul 31, 2026, 10:00 JST

Claude Code: 86% of token costs hide in invisible overhead, not your prompts

Claude Code: 86% of token costs hide in invisible overhead, not your prompts

3 Key Points

  1. What happened

    A real-world trace of Claude Code use across a 45-person team over 30 days found that only 14% of input tokens came from user prompts; the rest was hidden context like prior conversation history (30–45%), tool schemas, and system instructions. Uber saw per-person bills climb to $500–$2,000 per month across roughly 5,000 engineers, and one developer on a $200 Max plan consumed $50,000 in a single month.

  2. Why it matters

    Claude Code has already crossed $1 billion in annualized revenue, but most teams have no visibility into what drives their bills. The Anthropic Console, /cost command, and /context tool each show only partial information—none connect token breakdown to configuration choices to actual savings. Without that link, teams cannot tell whether an MCP server is worth its cost, whether to keep a 10,000-token CLAUDE.md file, or why bills are climbing.

  3. What to watch

    Comet's Cost Intelligence (integrated into Opik) intercepts Claude Code API traffic, maps every token to its source category, and surfaces specific recommendations—switch to Sonnet (estimated 22–48% savings), compact context sooner (10–15%), lower thinking effort (6–7%), or block unused MCP servers. The tool applies changes directly via "Apply" buttons or org-wide managed settings. Comet's own team cut median output cost by 21% (from $229 to $181 per million tokens) after refactoring configuration, with no change in development speed.

Not sure about something? Ask the AI

Questions and answers are published on this page.

Summaries like this, in your inbox every morning.

Context & Analysis

Claude Code's cost structure is opaque by design. Every turn includes the full conversation history, all MCP server schemas, and the system prompt—but none of this is visible in the Anthropic Console or the /cost command. A trace of real usage showed that prior assistant context alone (replies and tool outputs replayed from earlier turns) consumed 30–45% of input spend, yet most developers assume costs are driven by their prompts and the code Claude generates. The problem compounds at scale: Uber deployed Claude Code to roughly 5,000 engineers and saw bills climb to $500–$2,000 per month per person, while a single developer on a $200 Max plan burned $50,000 in one month—numbers that suggest widespread configuration drift rather than model cost alone.

The root cause is that configuration decisions accumulate invisibly. A developer adds an MCP server for one project and never removes it; CLAUDE.md files grow to 5,000–10,000 tokens over months; thinking effort defaults to "high" (later changed to "medium" after SWE-bench data showed 76% fewer output tokens at medium effort with the same completion rate). None of these choices show up in the org-level console. Checkmarx and other supply-chain scanning tools track API imports, but Claude Code's context is not a lockfile—it lives in configs, datasets, and session state, so security and cost teams track fragments instead of a unified view.

Comet's Cost Intelligence closes this gap by intercepting API traffic, mapping every token to its source category (prior assistant context, tool results, model choice, thinking effort, MCP schemas), and surfacing recommendations with estimated savings. By connecting attribution (which categories eat tokens) to policy (what should change) to implementation (applying changes org-wide), teams can move from blanket restrictions (which hurt productivity) to informed decisions—e.g., switching default model to Sonnet (which costs 0.6× Opus per tier and handles comparable development work) or blocking MCP servers with zero calls that still load schemas every turn.

FAQ
What percentage of Claude Code input tokens actually come from my prompts?
Only 14% of input tokens were actual user prompts in the traced deployment. Prior assistant context (conversation history and tool outputs) alone consumed 30–45% of total input spend.
How much can teams save by switching configuration?
Switching the default model to Sonnet can save 22–48% of spend; lowering thinking effort saves 6–7%; and compacting context sooner saves 10–15%. Comet's own team achieved a 21% reduction in median output cost (from $229 to $181 per million tokens) after refactoring.
What is included in the hidden 86% overhead?
Prior assistant context (78% from tool results, 22% from text), MCP server schemas (up to 16,000 tokens per turn with 10 servers and 50 tools), tool-related outputs (23% of input and 24% of output spend combined), thinking tokens (up to 40% of output spend at enterprise scale), and static overhead like system prompts and behavioral instructions.
Daily Dose of Data ScienceRead Original Article

AI news that matters for your work, delivered every morning.

Pick your industry and the AI tools you use, and get news related to your work every day.

Free · 30 seconds with Google · unsubscribe anytimeWhat is AIToday? →

Ask AI

Ask AI anything about this article. The AI reads this article, earlier AIToday articles, and Wikipedia, and cites its sources. Q&As are published on this page for other readers too.

Questions and answers are published on this page.

Related Articles

Next articleAmazon to spend $220B on AI infrastructure in 2026, but still faces capacity shortage