Kimi K3 lags US models on hacking tasks; distillation may explain gap

The evaluation reveals a significant performance gap between China's Kimi K3 and leading U.S. frontier models on offensive cyber tasks, even though Kimi K3 reportedly performs competitively on standard benchmarks. This asymmetry hints at how training methodology shapes model capabilities in ways that general performance metrics do not capture. The institutes' explanation—that Kimi K3 may have been built by distilling Claude outputs, which exclude advanced offensive cyber queries due to Anthropic's safety classifiers—suggests a structural reason for the gap rather than a simple capacity shortfall. U.S. frontier models were tested with their system-level safeguards disabled to measure maximum capabilities, exposing exploit skills that are normally inaccessible through public interfaces and therefore unavailable for distillation. This disparity matters because it shows how safety architectures in training data can inadvertently limit the transfer of certain capabilities during distillation.

Chinese open-weight models have been gaining cyber capabilities since early 2025, but according to CAISI's Elo-based time-series analysis, they remain 4–7 months behind leading U.S. systems—an improvement from the 6–10 month gap recorded at the start of 2025. The institutes note that this closing gap, combined with the fact that Kimi K3's safeguards do not prevent offensive cyber assistance, creates "a persistent and irreversible risk of misuse," particularly as open-weight models become more capable.