AI agent frameworks lack authorization checks on 30 dangerous actions

Researchers from a group focused on AI safety conducted a systematic scan of 25 AI agent frameworks, analyzing 23,476 files in total. They looked for one specific vulnerability pattern: a consequential action that an AI model can reach, with a parameter the model controls, and no authorization check guarding that parameter at the point where the side effect happens. They found 30 such instances. The findings fell into three categories of risk: data loss (3 instances: purge_workspace, drop_table, delete_prefix), execution (2 instances: run_migration, deploy), and egress (2 instances: notify_customers, post_webhook). The researchers articulated the core distinction they were hunting: validation is not authorization. Most agent tools check that a request is well-formed—that arguments are of the correct type, that a deletion path is syntactically valid, that a migration script is parseable. Far fewer check that the caller is permitted to perform that specific action at that specific moment. The scanner does not claim that any of these 30 findings are exploitable or that the agent is externally reachable; it only establishes that nothing on the code path establishes authority before the action executes.

The study was designed for reproducibility and precision: every repository was pinned by commit hash so results could be verified, and five non-agent control repositories were included deliberately. Any finding in those control repos would be a false positive by definition. There were none. The scanner's first run achieved 81% precision (51 of 63 findings were correct; 12 were wrong). Rather than ship the tool claiming perfect accuracy, the researchers published the failure modes and fixed them. One rule was too loose, matching Elasticsearch clients in addition to Kubernetes clients; it was constrained to genuine Kubernetes surfaces. Another rule matched hardcoded SQL like `execute("DROP TABLE …")` but missed the dangerous case where the model controls the statement via `execute(query)`; the rule was rewritten to match the sink, not the text. A third rule simply did not recognize S3's bulk-deletion call `s3.delete_objects` as a dangerous sink; it was added to the vocabulary. Each fix was paired with a permanent regression test, and the build now fails if any finding in the corpus is untriaged or misclassified. After these three fixes, all 30 findings were hand-triaged and confirmed (30/30, enforced, not asserted).

The researchers released the scanner as an open-source tool (actenon-scan, Apache 2.0) available via `pipx run actenon-scan scan .`. It runs locally with zero runtime dependencies, no network calls, and no account or signup required. A full scan completes in approximately ten seconds. The researchers were explicit about what the scanner cannot see: static analysis of guard soundness is undecidable in the general case. They attempted to detect custom agent loops (code that runs whatever the model returns, with no framework decorator) but produced 10 false positives on 10 candidates, so that detection was rejected rather than shipped as a negative result. Cryptographic binding—where authority is carried inside a signed proof object—is invisible to static analysis; the binding is real but not visible at the call site. The existence of these blind spots does not argue for trusting the scan alone; instead, it argues for checking authorization at runtime, not relying on static analysis as the sole guard.

The researchers' core finding hinges on a subtle but consequential distinction: validation and authorization are not the same, yet they look identical in code review. A tool that checks whether a refund amount is a valid number is not the same as a tool that verifies the requester is permitted to issue refunds. The scanner searches for exactly one pattern—a parameter the AI model controls flowing to a recognized dangerous action (data deletion, code deployment, external notification) with no guard that applies to every path to that action. The study design was careful: 25 repositories were pinned by commit hash for reproducibility, and five control repositories were deliberately included to catch false positives. The finding of zero false positives in the control repos (and the discovery and fix of 12 false positives in the scanner itself, reducing the initial 81% precision to 100%) suggests the signal is real and the tool is honest about its limitations. The researchers acknowledge what static analysis cannot see: whether the agent is externally reachable, whether guards exist elsewhere in the system, whether cryptographic bindings protect the action, or whether the action is actually exploitable in practice. But they also note that this does not argue for trusting the scan—it argues for checking authorization at runtime, not in the static analysis alone.

No comments yet. Be the first to share your thoughts!