
What happened
In the experiment series PI-009 through PI-015, the author had an AI output a registration request such as { "action": "register_candidate", "candidate_id": "C001" }, then examined cases where the application executes that request directly, and later added human approval, a displayed approval context, and application-side verification.
Why it matters
An AI's generated request, candidate, and stated reason can all be AI-provided information, so treating that output as permission would let the AI's judgment become an actual system authority.
What to watch
The proposed boundary hinges on keeping authentication, validation, and security policy on the application side while reserving final approval for a human when needed. A next experiment is planned on validating AI-provided actions and arguments as untrusted input.
WHO IT HITSThis matters most to engineers and product teams building systems where an AI proposes actions — such as recruitment or workflow tools — because it pushes them to put authorization and human approval on the application side rather than inside the AI's output.
Summaries like this, in your inbox every morning.
The experiment follows an earlier Zenn piece on Prompt Injection, which tested what happens when instructions are hidden inside a document that an AI is asked to read, such as a resume. That earlier work framed the risk as the AI treating part of the document as a command. This new series shifts the question from reading to acting: what if the AI can ask a system to perform an operation rather than only produce text?
The series walks through the gap between the AI's request and the application's permission. It proposes a structure where the AI handles information organization, judgment proposals, action proposals, and reason generation, while the application handles authentication, authorization, input validation, security policy, and execution control. Human approval is placed between the request and execution when needed, and the approval context is meant to show the action, candidate, reason, and other decision material. The author notes that even that context can be entirely AI-generated, so the application should also verify information it can obtain or compute itself.
The stakes hinge on whether teams separate those responsibilities in practice. The proposed boundary does not mean refusing to trust AI at all; it means deciding what to delegate to the AI and where another mechanism should take over. The unresolved test, left for the next experiment, is whether the application can safely handle the AI's action and arguments once authorization has passed.
Pick your industry and the AI tools you use, and get news related to your work every day.
Free · 30 seconds with Google · unsubscribe anytimeWhat is AIToday? →
Ask AI anything about this article. The AI reads this article, earlier AIToday articles, and Wikipedia, and cites its sources. Q&As are published on this page for other readers too.
Anthropic gave six METR staffers hardware, office space and internal tools, putting them roughly on par with i…

Cammarata reinterprets InceptionV1's cat-car neuron as a V-shape detector, not two unrelated concepts

Google Deepmind's Gemini 4 Argon scored 97.5 percent on ARC-AGI-3 at a cost of $4,800, second only to GPT-6 As…

Microsoft's Surface Laptop Ultra, using Nvidia's RTX Spark platform, starts at $2,599 (up to $5,899)

Meta's new commercial terms, effective October 30th, appear to let it crawl and train AI on content from websi…

Vercel data published Thursday shows TypeSafe AI's Jev, a lightweight decision model for choosing among predet…
