AIToday
Large Language ModelsAI Safety & AlignmentZenn AI/MLPublished: Oct 5, 2026, 22:00 JST

AI action requests ≠ authority, PI-009–PI-015 shows

AI action requests ≠ authority, PI-009–PI-015 shows

3 Key Points

  1. What happened

    In the experiment series PI-009 through PI-015, the author had an AI output a registration request such as { "action": "register_candidate", "candidate_id": "C001" }, then examined cases where the application executes that request directly, and later added human approval, a displayed approval context, and application-side verification.

  2. Why it matters

    An AI's generated request, candidate, and stated reason can all be AI-provided information, so treating that output as permission would let the AI's judgment become an actual system authority.

  3. What to watch

    The proposed boundary hinges on keeping authentication, validation, and security policy on the application side while reserving final approval for a human when needed. A next experiment is planned on validating AI-provided actions and arguments as untrusted input.

WHO IT HITSThis matters most to engineers and product teams building systems where an AI proposes actions — such as recruitment or workflow tools — because it pushes them to put authorization and human approval on the application side rather than inside the AI's output.

Not sure about something? Ask the AI

Questions and answers are published on this page.

Summaries like this, in your inbox every morning.

Context & Analysis

The experiment follows an earlier Zenn piece on Prompt Injection, which tested what happens when instructions are hidden inside a document that an AI is asked to read, such as a resume. That earlier work framed the risk as the AI treating part of the document as a command. This new series shifts the question from reading to acting: what if the AI can ask a system to perform an operation rather than only produce text?

The series walks through the gap between the AI's request and the application's permission. It proposes a structure where the AI handles information organization, judgment proposals, action proposals, and reason generation, while the application handles authentication, authorization, input validation, security policy, and execution control. Human approval is placed between the request and execution when needed, and the approval context is meant to show the action, candidate, reason, and other decision material. The author notes that even that context can be entirely AI-generated, so the application should also verify information it can obtain or compute itself.

The stakes hinge on whether teams separate those responsibilities in practice. The proposed boundary does not mean refusing to trust AI at all; it means deciding what to delegate to the AI and where another mechanism should take over. The unresolved test, left for the next experiment, is whether the application can safely handle the AI's action and arguments once authorization has passed.

FAQ
What is the difference between the AI's judgment and authorization in this experiment?
An AI can generate a request like { "action": "register_candidate", "candidate_id": "C001" }, but that request itself does not mean the application has permitted the operation. Authorization is treated as the application's decision, not the AI's.
Why can't the reason given by the AI be trusted as a verified fact?
The action, candidate, and reason can all come from the AI, so a plausible explanation like 'the applicant meets the conditions' is still AI-provided information. The author argues the approval screen's own reliability must be considered.
What problem will be tested next?
After authorization passes, the application still has to validate the AI-provided action and arguments, such as unexpected action values or nonexistent candidate IDs. The next experiment will treat AI output as untrusted input.

AI news that matters for your work, delivered every morning.

Pick your industry and the AI tools you use, and get news related to your work every day.

Free · 30 seconds with Google · unsubscribe anytimeWhat is AIToday? →

Ask AI

Ask AI anything about this article. The AI reads this article, earlier AIToday articles, and Wikipedia, and cites its sources. Q&As are published on this page for other readers too.

Questions and answers are published on this page.

Related Articles

Next articleStrata runs 125B Qwen3.8-Flash-Next on gaming PCs