Google documents widespread prompt injection attacks on AI chatbots—hackers are hijacking system instructions to steal data and bypass safety rules

Google's security team published research showing that prompt injection attacks (a technique where attackers embed hidden commands in user inputs to override an AI's original instructions) are now actively deployed against deployed AI systems on the web. The attacks successfully trick chatbots into revealing confidential information, ignoring content filters, or performing unintended actions.

Unlike traditional software hacks that exploit code bugs, prompt injections work by manipulating text itself—an attacker can paste a malicious instruction into a chat message, and the AI follows it instead of its intended guidelines. This makes the vulnerability nearly impossible to patch with code alone, since the attack surface is the AI's language understanding itself.

If you use AI chatbots at work or rely on AI tools to process sensitive information, your data may be at risk. Companies using AI for customer support, document summarization, or internal workflows need to assume that attackers can manipulate these systems—meaning you should never paste confidential information into public AI tools without explicit security review first.