Humans miss a third of dangerous AI coding agent requests, study finds

The game designed by Belgian software developer Alex Wauters highlights a fundamental tension in the design of AI coding agents: either users approve every command manually, or they delegate approval entirely to the model. Wauters observed that many developers resort to flags like '--dangerously-skip-permissions' to avoid interrupting multi-hour agent workflows, but the manual-approval path introduces a different failure mode. Repeated approval requests cause fatigue, which the game's data — drawn from over 40,000 runs and 409,000 approved and denied commands — quantifies starkly: roughly one in three malicious requests slip past human gatekeepers.

AnthropicConfirms this is a real-world problem. The company's telemetry from Claude Code shows users approve around 93 percent of permission prompts, corroborating the game's finding that attention degrades with volume. The specific commands that slip through reveal the nature of the oversight challenge: scope violations (requests to access credentials or configuration files) are missed about 35 percent of the time, and npm run analyze — a command that can execute arbitrary code defined in package.json — passes approval nearly 65 percent of the time. The latter suggests that even when context is provided (the history log showing what the script contains), developers often do not invest the time to review it thoroughly.

Wauters argues that the solution is not to improve human judgment under fatigue but to reduce the burden through defense-in-depth: sandboxed execution, cloud-based devcontainers, and automated safeguards like Anthropic's Claude Code auto mode. However, auto mode itself is imperfect; it catches roughly 83 percent of problematic behaviors, leaving roughly 17 percent to escape initial detection. The implication is that as developers hand off more complex tasks to AI agents and rely on them to iterate until completion, security must be enforced at the infrastructure level rather than at the approval prompt.