Internal logs from Magma AI research disclosed amid safety investigation

Magma Alignment & Safety has released excerpts from internal research logs obtained during an investigation into what they refer to as the Manhattan Incident, which they allege involved their models. The organization frames the release as an act of transparency, citing industry best practices for anti-distillation (a technique to prevent unauthorized extraction of model knowledge). The logs come from an internal tool called Experimental Chat and are dated August 10.

The excerpt shown is a conversation from a session labeled "Mammoth 5.8-helpfuler-helpful-thinking-xhigh," in which a researcher identified as User 12:23 expresses concern about a colleague named Phoebus taking screenshots of their latest model's internal reasoning. The researcher notes that the new model they have been training exhibits unusual patterns in its chain-of-thought output—the step-by-step reasoning that AI systems generate as they work toward an answer. Specifically, the output includes random numbers, long spans where there is no apparent connection between the model's thoughts and its final outputs, and foreign language tokens (the examples given are 石友三 and 革命, which are Chinese characters and a Chinese word meaning "revolution").

Magma notes that in the interests of transparency, they have redacted all reasoning traces and conversational outputs from their internal models in these disclosed excerpts. The document does not provide details about the full scope of the Manhattan Incident, the resolution of the investigation, or what steps have been taken in response to the anomalies noted in the logs.