GitHub explores validation framework for autonomous agents where correctness is outcome-based rather than step-by-step

GitHub describes how traditional testing breaks down for autonomous agents like Copilot Coding Agent when they navigate real environments (UIs, browsers, IDEs). The agent may succeed at a task while the test fails—a 'false negative'—because execution paths vary due to timing, loading screens, or environmental noise, even though the essential outcome is correct.

The proposed approach uses dominator analysis (a concept from compiler theory) to classify agent behavior into essential states (milestones required for success), optional variations (incidental states like loading spinners), and convergent paths (different sequences that reach the same outcome). Executions are modeled as graphs (Prefix Tree Acceptors) rather than linear scripts, allowing the framework to distinguish between incidental noise and critical failures.

GitHub argues that correctness for agentic systems must shift from 'did this happen in exactly this sequence?' to 'what had to happen for success to be real?' This reframing aims to reduce false negatives and move agents from experimental demos to production-grade infrastructure by validating structured behaviors rather than rigid execution paths.