Hugging Face attacked by AI agents; switches to open-weights GLM 5.2 model to defend

On July 16, Hugging Face disclosed a cyberattack on part of its infrastructure involving AI-driven agents. The attack exploited data pipelines on the platform and compromised some datasets, service billing records, and user account information. Hugging Face's security team initially attempted to defend using AI, but encountered a critical barrier: commercial models available through standard APIs refused to analyze attack patterns and payload details because those requests violated the models' safety guardrails designed to prevent misuse.

The attack mechanism was sophisticated. Malicious data appeared in data pipelines used by Hugging Face; attackers then ran code using a template injection technique to execute a second piece of code across data loaders and dataset templates. This multi-stage approach allowed them to move between clusters and exfiltrate data over several days. The attackers used an LLM to perform initial reconnaissance, though the specific model was not disclosed.

Faced with the inability to use commercial AI for real-time forensic analysis, Hugging Face deployed Zhipu.ai's open-weights model GLM 5.2 directly on its infrastructure. Open-weights models, which are publicly released without proprietary restrictions, can be queried directly by their host without the safety filters applied to commercial API models. This enabled Hugging Face to analyze attack behavior and mount a defense without delay.

The company's response measures included audit log review, isolation of compromised nodes, reconstruction of affected infrastructure, deployment of updated guardrails, and role-based access controls. Hugging Face also notified affected users and reported the incident to authorities. The CEO commented that while attackers are constantly evolving their techniques, the company did not believe attackers' data or code would leave the platform, though this depends on proper deployment of host-side model safety measures. Hugging Face has been providing security research tools and gathering feedback from cloud security researchers as part of its ongoing incident response effort.

The incident reflects a fundamental tension in AI security. Commercial AI providers—OpenAI, Anthropic, and others—have built safety guardrails into their models to prevent misuse, including analysis of exploit techniques and attack payloads. These safeguards are intentional and well-justified for general use. However, when a cloud platform's own infrastructure comes under attack, real-time forensic analysis of attacker behavior becomes a critical business function, and safety guardrails can block the very queries needed to mount an effective defense.

Hugging Face's turn to an open-weights model reveals the trade-off organizations face. Open-weights models deployed on a company's own servers can be queried without restrictions, enabling rapid incident response—but at the cost of losing the safety oversight that commercial models provide. The company's reliance on Zhipu.ai's GLM 5.2, a model without commercial API restrictions, allowed faster analysis but also highlights that organizations managing sensitive infrastructure may need to maintain security-focused AI tools separate from public-facing services.

No comments yet. Be the first to share your thoughts!