Claude and Codex Reviews of Same Code Agree on Nothing

The Zenn author's starting question was whether using multiple AI agents genuinely widens the perspectives brought to a problem. Prior work the article cites suggests simply adding more agents does not guarantee this: even different LLMs can make correlated errors, and adding homogeneous agents hits performance saturation quickly.

The trial itself borrowed a framing from software-inspection research, where the overlap between findings by different human inspectors is used to estimate how many latent defects remain undiscovered. Replacing inspectors with AI reviewers, the author ran Claude and Codex in blind, independent passes over the same code and specification. In the initial EXP-001-R2 run, each reported one valid finding with no overlap between them; across three targets, the 7 valid findings included 6 reported by only one reviewer. In one case, one reviewer passed the code while the other flagged three issues, including a stale-state race condition.

The author is careful not to claim victory for diversity. Non-overlapping findings could arise simply because latent defects are numerous and each reviewer's per-run detection rate is low — the same model run twice would also produce different results. The planned next step is to measure the natural variance of repeated identical runs as a baseline, then check whether swapping the model, role, or context produces a reliably larger increment. The value of the whole framework, the author suspects, may ultimately lie less in code review and more in upstream problem framing, where an agent locked onto one interpretation of a customer request could otherwise spend its effort solving the wrong problem correctly.