Google says it stopped a zero-day exploit developed with AI for the first time

Google Threat Intelligence Group detected 'prominent cyber crime threat actors' planning to use a zero-day vulnerability in an unnamed open-source web-based system administration tool to bypass two-factor authentication in a 'mass exploitation event.' Google's researchers found evidence of AI involvement in the Python script, including a 'hallucinated CVSS score' and 'structured, textbook' formatting consistent with LLM training data.

The vulnerability exploited 'a high-level semantic logic flaw where the developer hardcoded a trust assumption' in the platform's 2FA system. Google was able to 'disrupt' this particular exploit, though the report notes researchers 'do not believe Gemini was used' in the attack.

Google's report indicates hackers are increasingly using AI to find and exploit security vulnerabilities, including through 'persona-driven jailbreaking' (instructing AI to roleplay as a security expert) and feeding AI models entire vulnerability repositories to refine AI-generated payloads before deployment.