Pangram Labs has developed an AI text detector that achieves 99.64% accuracy on current AI model outputs, substantially outperforming existing tools on humanized and adversarially modified text. The company has released an open-source version and now provides confidence scores rather than binary verdicts, addressing the growing challenge of detecting synthetic text as language models become more sophisticated.
Summaries like this, in your inbox every morning.
Sign up free →What happened
Pangram Labs, a team of over 25 people, has built an AI text detection system that achieves 99.64% accuracy on detecting outputs from Fable 5 (a recent AI model). The company has also released an open-source model based on Llama-3.2-3B and shifted its production classifier from binary verdicts to percentage-based confidence scores.
Why it matters
Detecting AI-generated text is becoming harder as language models improve and users learn to humanize outputs. Pangram's detector significantly outperforms existing alternatives (GPTZero and Binoculars) on adversarially modified AI text, reaching 93.66% accuracy where competitors achieve under 35%, which may help platforms and educators identify synthetic content more reliably.
What to watch
The company is actively engaged on Twitter (handle @pangram, affiliates tab available for direct engagement). The open-source Llama-3.2-3B QLoRA model was state-of-the-art at release, though the body does not specify a release date.
Pangram Labs addresses a critical gap in AI safety and content authenticity as language models become increasingly indistinguishable from human writing. The body notes that their production model now detects Fable 5 outputs with 99.64% accuracy, a significant improvement over the adversarial benchmarks shown in their research paper. The shift from binary classification to percentage-based confidence scores reflects a more nuanced approach to detection, allowing users and platforms to set their own thresholds based on risk tolerance.
The company's release of an open-source detector based on Llama-3.2-3B extends access beyond enterprise customers and may accelerate adoption in academic and non-commercial settings. The body indicates that testing the open-source model on adversarially modified text may trigger AI safety safeguards, hinting at the sophistication and potential risks of the detection task itself. With a team of over 25 full-time employees and active engagement on social media, Pangram appears positioned to scale detection capabilities as new AI models emerge.
No comments yet. Be the first to share your thoughts!
Log in to join the discussion
Get curated AI news from 200+ sources delivered daily to your inbox. Free to use.
Get Started FreeFree · takes 30 seconds · unsubscribe anytime
1 minute a day. The AI essentials.
200+ sources · Email / LINE / Slack
