AWS PII detector built on LLMs hits 81.6 percent with OSS-GPT 20B

The detector addresses a common problem in fine-tuning: training corpora are full of PII, and a model trained on uncleaned text can memorize and later reproduce that data. Off-the-shelf tools are bi-directional token-classification models with a fixed PII schema and locked to one model and one deployment. Adding a new entity type means relabeling and retraining.

The LLM-based approach reframes detection as configuration. The entities to detect, the output format, and the deployment backend all live in the prompt or a thin interface, so one detector can target a new entity type by editing a prompt instead of retraining. It scores at or above frontier models on high-stakes identifiers such as SSN, financial, and ID numbers (all above 95 percent), though DATE remains a shared weak spot at about 50 percent.

The outcome hinges on whether teams value that flexibility over the simplicity of a fixed-scope tool. For organizations that need to detect domain-specific identifiers or run in secure environments, the ability to change the entity set with a one-line prompt edit may matter more than a small difference in accuracy.