Harnesses Now Outperform Models in AI Coding Benchmarks

Endor Labs conducted a comparative study of coding performance across two frontier models and two harnesses, revealing a surprising inversion of conventional wisdom: the harness now shapes model output more than the model itself. OpenAI's GPT-5.5 achieved 61.5% functional correctness when run through its native Codex harness, but when the same model was tested through Cursor's harness in the same week, it scored 87.2%—a 25.7-point swing attributable solely to the runtime interface. Similarly, Anthropic's Claude Opus 4.7 scored 87.2% functional correctness in Claude Code (Anthropic's native harness) and 91.1% in Cursor. Both frontier models performed better in a competitor's harness than in the one their maker ships. This performance gap centers on cost control through intelligent caching and context selection. Input tokens constitute 86-98% of LLM traffic on OpenRouter; while output costs 5× per token compared to input, input volume is so large that it dominates total expenditure—practitioners report a 95% input / 5% output cost split at scale. Because the model does not control which context gets sent or cached, but the harness does, input cost management is now the harness's domain. A study across 500 long-horizon agent sessions found that harnesses employing smart caching strategies—such as caching only the stable prefix and placing dynamic content after the cache breakpoint—achieved 41-80% cost reduction and 13-31% faster time-to-first-token. These savings scale linearly from 500-token to 50,000-token prompts. Cursor's harness employs three key techniques: dynamic tool fetching (retrieving only necessary code or documentation), priority-based prefix assembly (ordering context by importance), and two-tier caching (maintaining both stable and dynamic content layers). These are precisely the same techniques used in Claude Code, which reports a 96% cache hit rate in real sessions and leverages user-shared system-prompt caching and byte-identical sub-agent forking for 90% savings. The fact that Cursor matches or exceeds these results without being Anthropic's first-party harness suggests that the discipline enabling high cache hit rates lives in the harness architecture, not in privileged access to the model. First-party co-design of harness, cache API, and model does buy real cache discipline, but that discipline can be replicated by a sufficiently sophisticated third-party harness. This finding repositions harnesses from mere wrappers around models into primary performance levers—what the article calls a jockey pushing AI further than the breeder imagined.

The findings challenge a long-held assumption in the AI stack: that model selection dominates performance and cost. Endor Labs' testing reveals that the harness—the software layer orchestrating how a model receives and processes queries—now exerts greater leverage. Input tokens constitute 86-98% of LLM traffic on OpenRouter, and because output costs 5× input per token but input volume overwhelms the ratio, controlling input costs is where economic value concentrates. The model itself cannot control which context is cached or sent; that power belongs to the harness. Cursor's ability to extract higher functional correctness scores from both GPT-5.5 and Claude Opus 4.7 than their native harnesses demonstrates that this leverage is not locked to first-party integration. Techniques like dynamic tool fetching, prefix assembly based on priority, and two-tier caching—all harness-level choices—can replicate or exceed the cache hit rates (96% in Claude Code's real sessions) that model makers achieved through deep co-design. For practitioners at scale, this means the harness decision may now rival or surpass the model decision in determining both cost and performance.

No comments yet. Be the first to share your thoughts!