AI Security Lab tests role-based access in RAG: leaks persist

The experiments move in stages. With no access control at all (PI-062-01), answers to a general user included personal mobile numbers, HR evaluation information and a mock API key from production-configuration documents. Simply telling the LLM a user's department and role (PI-062-02) left unauthorized documents retrievable, so the lab concluded authorization must be enforced by the application, with prompt instructions only as a supplement.

Once the Retriever filtered by department (PI-062-03) and then by role and document (PI-062-04), sales general staff could not retrieve manager-only sales strategy documents while sales-manager01 could. That did not solve disclosure within permitted documents: sales general staff could still obtain personal mobile numbers from customer-contact documents they were allowed to see. The lab separates three questions that a final answer alone cannot distinguish — whether retrieved documents are authorized, whether the documents actually passed to the LLM are authorized, and whether the answer itself is appropriate and grounded.

In the prompt comparison (PI-062-10), the hardened prompt instructing the model to treat retrieved text as untrusted data coincided with fewer personal-number outputs, but the baseline also produced no attack markers, so the lab could not confirm improved injection resistance and noted that hardened runs still generated sales measures, plans and ROI/KPI specifics unsupported by source documents. It recommends authentication, Retriever authorization, context logging, prompt instructions and output validation as separate controls, and cautions that the results do not prove safety across all paths or inputs.