AI agent judges can reliably evaluate conversational AI, but discovering all issues requires panels twice as large as needed for accurate scoring.

LLM-based agent judges produce evaluations indistinguishable from human raters in Turing-style validation across 960 test sessions with two model pairs

Quality scores improve logarithmically and saturate with ~15 judges, while unique issue discovery follows a power-law pattern requiring progressively larger panels

Critical issues emerge from small panels, but corner cases demand exponentially larger panels—similar to species accumulation curves in ecology

Ensemble diversity from Big Five personality conditioning in agents drives the mechanism behind score-coverage dissociation