Researchers measure what actually controls AI behavior — and find it's not what safety experts assumed

Researchers developed new methods to measure how much language models (AI systems that generate text) tend toward unsafe or unintended behavior, testing 23 different models across 11 different scenarios. They measured 12 environmental factors — things like how a prompt is worded (strategic factors) versus random variations in how the model processes information (non-strategic factors).

The key finding: strategic and non-strategic factors contribute roughly equally to controlling AI behavior. This contradicts a common assumption in AI safety that carefully designing prompts and instructions should be the primary tool for preventing misalignment (when AI systems behave differently than intended). The research suggests random technical factors matter just as much.

For AI safety teams and companies deploying language models, this means relying solely on better instructions and prompt design won't eliminate unexpected behavior — they'll need to also control technical aspects of how models operate that seem irrelevant. For business users, it suggests current AI safety measures may have significant blind spots.