AI model's learned token diverges from intended meaning

Researchers trained a new token on data generated by a model steered with a persona vector, then tested whether the model could explain what it had learned. The model's responses using the token showed higher alignment with the steering vector than direct steering, but the model's own explanations of the token's meaning diverged from the intended persona—calling it "dread" when it was meant to represent "evil," or "warmth" instead of "sycophancy."