Open-weight AI models now match top systems from 4 months ago, at fraction of cost

The British AI Security Institute released the first public assessment of how far leading open-weight AI models lag behind proprietary frontier systems in cyber capabilities, and the findings paint a picture of rapidly narrowing technical gaps and dramatic cost disparities that create new risks for defenders.

AISI tested two leading open-weight models, GLM-5.2 (released June 2026) and DeepSeek V4-Pro, using two distinct benchmarks. The first, Narrow Cyber Tasks, consists of 70 tasks across four difficulty levels—from nontechnical work to expert-level challenges—covering vulnerability research, reverse engineering, web exploitation, and cryptography. GLM-5.2 matched the performance of Opus 4.6 (released February 2026) on these tasks, placing it approximately four months behind the frontier. DeepSeek V4-Pro performed at the level of Opus 4.5, released in November 2025. The second method, Cyber Ranges, tests autonomous cyber capabilities in simulated networks. One test, called "The Last Ones," simulates a 32-step attack on a corporate network with four subnets and about 20 hosts, which AISI estimates would take a human expert roughly 20 hours to complete. GLM-5.2 performed about as well as Opus 4.5 in this test, while DeepSeek V4-Pro fell below Sonnet 4.5. GPT-5.6-Sol and Claude Mythos 5 performed best. The gap on Cyber Ranges was wider, at around seven months, though AISI treats this result as weaker evidence because it comes from fewer test scenarios.

The performance gap has tightened significantly. For most of 2025, open models lagged by six to ten months; current open models now reach a level that closed models hit four to seven months earlier. This acceleration gained urgency in April 2026 when two closed models, Mythos Preview and GPT-5.5, delivered what AISI describes as "some of the largest gains in AI cyber capabilities since AISI began testing," prompting the UK's National Cyber Security Centre to issue international warnings about a rapidly changing cyber threat landscape.

The cost difference between open and closed models is stark. A 100-million-token Cyber Range test cost approximately $85 with Opus 4.5 or 4.6, roughly $46 with GLM-5.2, and just $1.19 with DeepSeek V4-Pro. For individual tasks that both models solved reliably, Opus 4.6 cost about $15 per task, GLM-5.2 cost around $6, Opus 4.5 cost about $12.50, and DeepSeek V4-Pro cost just 28 cents. These cost differences, combined with the ability to download, modify, and run open models without oversight, make scaled cyberattacks cheaper and easier to execute.

AISI found that safety measures on open models are largely ineffective. DeepSeek V4-Pro sometimes refused reverse-engineering tasks, but simply trying again was enough to bypass the restriction. The core problem is structural: safeguards like monitoring, classifiers, and user limits cannot reliably carry over to open models because they depend on the provider controlling access. Once a model's weights are public, users can remove guardrails, share copies freely, and run the model on private systems beyond anyone's control. AISI calls this "a persistent and irreversible risk of misuse." A recently published study found that terrorist groups are also jailbreaking commercial chatbots to plan attacks, showing that safety bypass is not unique to open-weight models, but open models' unrestricted availability adds another layer of risk.

Despite these concerns, AISI acknowledges the benefits of open-weight models: users can host them privately without data flowing back to providers, customize them for their needs, reduce costs significantly, and rely on a foundation that providers cannot unilaterally change or shut down. The institute says these competing concerns need to be balanced. From a defense perspective, the narrowing gap creates what AISI calls a "window for preparation"—a period during which cyber defenders with access to the strongest closed systems can act before equivalent capabilities become freely available without comparable safeguards. Recent gains have made that window more urgent. AISI plans to test Kimi-K3, whose weights are due out in late July 2026; current coding benchmarks suggest it could come closer to today's frontier models, though at a much higher cost than other open models.

The British AI Security Institute's assessment marks a watershed moment in the open versus closed AI model debate. The performance gap in cyber capabilities has collapsed from six to ten months at the start of 2025 to four to seven months today—a trend that accelerated notably in April 2026 when two closed models, Mythos Preview and GPT-5.5, delivered what AISI calls "some of the largest gains in AI cyber capabilities since AISI began testing." This prompted the UK's National Cyber Security Centre to issue international warnings that the cyber threat landscape is changing fast.

The cost economics amplify the urgency. DeepSeek V4-Pro costs roughly 1.4% of Opus 4.5 per 100-million-token test ($1.19 versus $85), and for individual solved tasks it costs 2.2% as much (28 cents versus $12.50). At that price point, the barrier to entry for scaling cyberattacks drops dramatically. Equally important, AISI's research shows that the safety guardrails baked into open models offer almost no practical protection—they are designed to work only when the provider controls access to the model, a condition that ceases to exist the moment weights are released publicly. Once released, users can remove guardrails, modify the model, and run it on private systems beyond any oversight.

AISI acknowledges this creates what it calls "a persistent and irreversible risk of misuse," but the institute also notes that open models offer genuine benefits: private hosting without data flowing to providers, customization, cost savings, and independence from provider control. The key implication for defenders is that the window of time to prepare—using the strongest closed models before equivalent capabilities become freely available—is now narrower and more urgent than it was six months ago.