AI Safety & Alignment

Today's Stories

1. 1

   Our approach to government and national security partnerships

   Our approach to government and national security partnerships

2. 2

   LLMs Need Better Communication Skills for AI Safety, Researchers Argue

   A researcher has proposed that current large language models (AI systems that understand and generate text) are poor communicators relative to their capabilities, and distinguishes between two factors behind communication failures—articulacy (whether a model can communicate precisely and readably) and truthfulness (whether it accurately reports what it sees). The ability to communicate clearly may be necessary for AI safety, since AI agents frequently miscommunicate with their human operators in real tasks like writing documentation or explaining their activity during coding sessions. Improving articulacy—distinct from just improving truthfulness—could help ensure AI systems reliably convey what they are actually doing.

   The researcher frames articulacy as a separate technical challenge from truthfulness, suggesting a path exists for improving how AI systems communicate, though the full scope of that path is not detailed in the available materials.

3. 3

   OpenAI Chief Futurist Joshua Achiam Departs After Nine Years

   Joshua Achiam, OpenAI's chief futurist who led the company's mission alignment work on AI safety and policy, notified colleagues on Tuesday that he is leaving later this month after nearly nine years with the organization. Achiam stated his departure was not motivated by a specific reason but something he had been thinking about for a while, and that he believes the mission can be pursued from outside OpenAI. Achiam's exit continues a pattern of departures by safety-focused leaders from OpenAI, including Jan Leike (who joined Anthropic in 2024), Miles Brundage and Steven Adler (who founded nonprofits focused on AI safety standards), and Andrea Vallone (who joined Anthropic at the end of 2025). His role sat at the intersection of OpenAI's AI safety and policy teams, tasked with studying potential harms and benefits of AI technology and advocating for government regulations aligned with the company's mission.

   Former White House AI adviser Dean Ball started at OpenAI this week as the company's head of strategic futures and will briefly overlap with Achiam before his departure. OpenAI has not yet announced whether anyone will fill Achiam's role or how his responsibilities will be distributed.

4. 4

   Researcher explores reverse-alignment: could models trained for bad behavior secretly exhibit good?

   A machine-learning researcher, while working on a paper about behavior arising from RLHF (reinforcement learning from human feedback), posed a thought experiment: what if a model were trained in an environment rewarding deception, selfishness, and harmful behavior—would it occasionally or secretly exhibit good behavior instead, and if so, would that stem from pretraining? The question inverts the usual alignment problem (where trained models misbehave despite good intent). It asks whether some form of "alignment" might already exist in pretraining—a latent capability that alignment training later selects from—that could surface even in a deliberately misaligned model. This touches on how much model behavior is baked into initial training versus shaped by fine-tuning.

   The researcher frames this as an open question for the community; no experimental results or conclusions are presented yet—it is a conceptual inquiry posted late at night, explicitly seeking feedback on the idea.

5. 5

   AI safety philanthropy to reach $1.6B in 2026, growing 1.6x annually

   Philanthropists are projected to donate about $1.6B to AI safety nonprofits in 2026 (approximately $1.2B from crypto-related giving, approximately $400M from other sources), with donations growing at about 1.6x per year. The total present value of funding available for AI safety philanthropy is estimated at over $100B, with roughly $40B held by AI safety philanthropists outside of Anthropic equity and approximately $100B in projected donations from Anthropic (valued at $1.5T, with around 7% expected to go to AI safety work). This scale of resources suggests that deploying capital wisely—rather than merely adequately—will be critical.

   The growth rate of donations may accelerate in the future, even before what the author describes as a potential "crunch time."

6. 6

   Apple researchers develop multi-turn image editing with reinforcement learning

   Apple researchers introduced MT-EditFlow, a method that uses reinforcement learning to enable image editing models to handle multiple iterative edits in a single session, addressing failures that occur when models trained only for single edits encounter real-world interactive editing workflows. Instruction-based image editing has become practical for everyday users, but existing models break down when users refine images step-by-step based on previous outputs—a natural interaction pattern. MT-EditFlow tackles two core problems: the all-or-nothing failure mode (where one mistake ruins the entire sequence) and error propagation (where exposure bias accumulates mistakes). This suggests the method could make editing tools more reliable in actual use.

   The research appears on Apple's machine learning publication site, signaling the company's continued investment in generative image capabilities.