NVIDIA: AI agent security is an engineering problem, not a model problem

NVIDIA's blog post is framed around a concrete failure case: an AI agent that updates customer information reads an attached document and follows malicious instructions embedded in it, trying to send customer data to an unapproved destination. That scenario is described as an example of prompt injection, where instructions are hidden in external documents or data to steer an AI's behavior. NVIDIA's answer is not to try to make the agent's judgment perfect, but to make sure that even if the agent follows the wrong instruction, network policy can block the data transmission, and the access decision and its result are recorded in logs.

The post also distinguishes between having a permission and needing it. An agent that is authorized to update customer information does not automatically need the right to send that information externally, and NVIDIA says there needs to be a mechanism where the agent can request additional permissions but cannot approve them itself. Responsibility for deciding whether the safeguards actually work should be assigned before production, and problems found in testing should lead to fixes.

NVIDIA positions all of this as a system design question rather than a model question. Its recommended measures include giving each agent a traceable ID, limiting credentials to the task at hand, defining in policy what data and systems the agent can reach and which actions need approval, requiring human approval for high-impact actions and permission changes even inside policy, verifying the origin and integrity of tools, skills and dependencies, and keeping protected records of tool calls and authorization decisions along with clear steps for revoking access and containing incidents.