What happened
Researchers from the UK AI Security Institute and alignment startup Timaeus have formed Sequent, a nonprofit research organization aiming to develop alignment techniques—methods to ensure AI systems remain safe and controllable—for superintelligent AI systems. Sequent plans to reach 40–80 fulltime employees within a couple of years and aims to raise $100–150M initially, with preparation to raise at least one order of magnitude more if successful.
Why it matters
Today's AI systems show occasional unpredictable failures, but humans can monitor and fix them. However, as AI systems become smarter and start handling more of their own development through recursive self-improvement, better alignment techniques will be critical to maintain control. Sequent argues that current frontier AI labs use methods that are "essentially reactive" and "do not yield principled insight into if or when they will fail." Independent organizations like Sequent may be better positioned to raise concerns if labs are taking unsafe approaches.
What to watch
Sequent's research will focus on multiple areas—scalable oversight (methods to monitor AI behavior at scale), learning theory, game theory, and personas—with the belief that interactions between these directions could yield breakthroughs. The organization intends to pursue "a portfolio of differentiated alignment bets" rather than a single approach, in contrast to the strategies of major AI labs.
Ask the AI about this article →
For example, today's edition would include:
AI-summarized, only the topics you pick — one digest a day via Email, Slack, or Discord.
Free · takes 30 seconds · unsubscribe anytimeWhat is AIToday? →
Ask AI anything about this article. Q&As are published on this page for other readers too.
The U.S. Department of Defense announced on August 31 that it has deployed ChatGPT Mil, a customized version o…

OpenAI stopped running inference on a model involved in the HuggingFace incident, but the post argues this is…

OpenAI announced its support for California Senate Bill 1119, which aims to establish strong, age-appropriate…

A UK study by UK AI Security Institute and Limbic AI surveyed 6,474 British adults

Anthropic trained an Opus-class model with large-scale reinforcement learning on environments vulnerable to re…

Broadcom's Clayton Donley says companies are doing mission-critical work with AI agents quickly, but without t…