AIToday
Zenn AI/MLPublished: Nov 16, 2026, 04:00 JST

AI Labo: parallel branches cut AI bug-fix tokens 67.2%

Send on LINE
AI Labo: parallel branches cut AI bug-fix tokens 67.2%

3 Key Points

  1. What happened

    In test E13, single-session retry used 4 model calls and 13,822 input tokens; parallel branches used 2 calls and 4,537 tokens, and both passed all 20 tests.

  2. Why it matters

    The result suggests that retrying repeatedly in one session inflates token use much faster than generating independent branches, which matters for teams managing AI agent costs.

Not sure about something? Ask the AI

Questions and answers are published on this page.

Summaries like this, every morning.

Context & Analysis

The experiment was designed around a realistic legacy codebase: three payment ledger modules (ingest.py, reconciler.py, summary.py) seeded with multiple bugs and an O(n^2) double-loop performance bottleneck. Mode A simulated the common developer habit of patching the same session over and over, while Mode B generated two clean-context branches from scratch — one conservative patch and one clean O(n) redesign — and let a deterministic judge pick the winner. The trace data shows that Mode A's input tokens grew from 2,377 in Turn 1 to 4,383 in Turn 4, a 1.84x increase, which the report identifies as the mechanism behind the token gap.

The judge's decision proved consequential. The conservative branch passed all functional tests but retained the double loop, scoring 80/100 and running the 2,000-event benchmark in 202.952 ms. The clean redesign scored 100/100 and ran the same benchmark in 0.962 ms. Without a deterministic judge weighing structural checks and benchmarks, the report argues, an LLM reviewer might have favored the smaller-looking conservative patch. The practical recommendation is to switch to parallel branches after two failed retries, before sunk-cost bias and context bloat take hold.

FAQ
What exactly is a 'deterministic judge' in this experiment?
It is a script that scores each branch mechanically using functional boundary tests, an AST check for double loops, and micro-benchmark measurements, then automatically selects the highest-scoring branch without human review.
How much did the parallel approach cost compared with the retry approach?
The parallel tournament cost $0.016453 in total, compared with $0.023238 for the four-turn single-session retry, a 29.2% reduction.
Did both methods produce equally fast code in the end?
No. Both passed all functional tests, but the parallel method's winning branch processed 3,000 events in 1.280 ms, while the single-session retry took 1.582 ms, a 19.1% difference.
Send on LINE

AI news that matters for your work, delivered every morning.

Pick your industry and the AI tools you use, and get news related to your work every day.

Free · 30 seconds with Google · unsubscribe anytimeWhat is AIToday? →

Ask AI

Ask AI anything about this article. The AI reads this article, earlier AIToday articles, and Wikipedia, and cites its sources. Q&As are published on this page for other readers too.

Questions and answers are published on this page.

Next articleNikkei survey: New York's 50 AI startups tied closely to finance, medical