Researchers crack encrypted AI reasoning, find passwords in leaked sessions

The vulnerability represents a fundamental breakdown in the security model that AI labs constructed around their reasoning processes. When Matthew Green, a cryptography expert, first reported in May that encrypted reasoning blobs could be replayed outside their original context, the providers dismissed the finding, saying they did not see security implications in side channels or replays. The new research from Panfilov's team demonstrates that assessment was incorrect—the portability of encrypted thoughts across sessions, users, and models within a single provider creates multiple avenues for misuse.

What makes the discovery particularly significant is its potential connection to the practice of "reasoning distillation," where a less capable model is improved by training on the reasoning outputs of a more powerful one. The researchers argue that it may have been possible for some time to extract reasoning processes for training proprietary models without breaking the cryptography itself. This supports mounting concerns that Chinese AI makers have been using reasoning traces to train their models on chain-of-thought data, a suspicion strengthened by analysis of Kimi-K3, which shows unusually high similarity to Claude and GPT reasoning segments.

The incident also exposes a gap between what AI models are actually doing and what their creators show users. The sanitized summaries presented in chat interfaces omit important details—models sometimes already know answers and then reverse-engineer plausible solution paths, or they get stuck in loops of nonsensical terms like "vantages," "marinades," and "watchers." Researchers at Arizona State University have warned that this humanized presentation creates false confidence in model controllability and may steer AI safety research in the wrong direction.