A harness is the software wrapper around an AI model, including tools, memory management, and rules that let the model act independently. It handles memory, context, and feedback for long-horizon tasks.

What benchmark did Nvidia test on and why does it matter?

Nvidia tested on ARC-AGI-3, an interactive reasoning benchmark of 2D games with no instructions where the model must figure out how to play and win. A 100% score means the model performs as well as humans—a result that has frustrated OpenAI, which scored less than 10% on the same benchmark.

What was the key innovation in Nvidia's harness?

Nvidia introduced a supervising agent component that acts like a CEO, nudging the main agent when it gets stuck or explores dead-end paths. Most agent users today rely on only one layer for their harness; Nvidia's approach uses two.

Large Language ModelsAI Coding AssistantsTop Companies' AI MovesTop Companies AIAug 22, 2026Aug 23, 2026, 06:31 JST

Nvidia shows AI harness, not model, is key to long-horizon tasks

Send on LINE

3 Key Points

What happened
Nvidia researchers used a custom harness with a "supervisor" component to get Claude Opus 5 to score 100% on the ARC-AGI-3 interactive reasoning benchmark, where the same model scored only 30% without the harness. A harness is the software wrapper around an AI model—tools, memory management, and rules that let it act independently.
Why it matters
The finding challenges the assumption that an AI model's underlying intelligence is what matters most for long-horizon tasks (those requiring many chained decisions over time). Databricks research showed in July that harness choice can 2× costs even when using the same model, meaning businesses may be optimizing the wrong layer. Nvidia's result also directly addresses a weakness OpenAI struggled with; OpenAI's own harness tweaks in recent research only tripled scores but did not approach 100%.
What to watch
Most agent users today rely on single-layer harnesses like Claude Code or Codex. Nvidia produces open harness components under the Nemo brand (some commercial, some freely available), giving users more control than closed systems. Nvidia emphasizes that open harnesses let teams adjust accuracy by controlling the harness, infrastructure, and runtime.

Not sure about something? Ask the AI

Questions and answers are published on this page.

Summaries like this, in your inbox every morning.

Get news like this every morning, free

Context & Analysis

Nvidia's research arrives at a moment when the field is shifting focus from raw model capability to the systems around models. The company's 100% score on ARC-AGI-3 is particularly significant because it directly challenges OpenAI's earlier struggles on the same benchmark—OpenAI scored less than 10% and was forced to conduct its own research in response. OpenAI's subsequent harness tweaks (adjusting two settings) did improve performance substantially by tripling scores, but still fell short of Nvidia's result, suggesting that the architecture of the harness itself, not just tuning, matters. Databricks' July research showing that harness choice can 2× costs even for the same model reinforces the finding that businesses optimizing purely for model selection are missing a major lever. Nvidia's framing of the harness as the critical piece also serves the company's commercial strategy: it positions open harness components (which Nvidia offers under the Nemo brand) as strategic assets, allowing users to avoid lock-in to proprietary systems like OpenAI's Claude Code or Anthropic's offerings.

FAQ

What is a harness?: A harness is the software wrapper around an AI model, including tools, memory management, and rules that let the model act independently. It handles memory, context, and feedback for long-horizon tasks.
What benchmark did Nvidia test on and why does it matter?: Nvidia tested on ARC-AGI-3, an interactive reasoning benchmark of 2D games with no instructions where the model must figure out how to play and win. A 100% score means the model performs as well as humans—a result that has frustrated OpenAI, which scored less than 10% on the same benchmark.
What was the key innovation in Nvidia's harness?: Nvidia introduced a supervising agent component that acts like a CEO, nudging the main agent when it gets stuck or explores dead-end paths. Most agent users today rely on only one layer for their harness; Nvidia's approach uses two.

AI news that matters for your work, delivered every morning.

Pick your industry and the AI tools you use, and get news related to your work every day.

Get it free

Free · 30 seconds with Google · unsubscribe anytimeWhat is AIToday? →

Ask AI

Ask AI anything about this article. The AI reads this article, earlier AIToday articles, and Wikipedia, and cites its sources. Q&As are published on this page for other readers too.