AIToday

New nonprofit Sequent launches to develop alignment techniques for superintelligent AI, raising $100–150M because current lab methods lack principled confidence in safety.

Import AI2d ago3 min read
New nonprofit Sequent launches to develop alignment techniques for superintelligent AI, raising $100–150M because current lab methods lack principled confidence in safety.

Summaries like this, in your inbox every morning.

Sign up free →

3 Key Points

  1. 1

    What happened: Researchers from the UK AI Security Institute and alignment startup Timaeus have formed Sequent, a nonprofit research organization aiming to develop alignment techniques—methods to ensure AI systems remain safe and controllable—for superintelligent AI systems. Sequent plans to reach 40–80 fulltime employees within a couple of years and aims to raise $100–150M initially, with preparation to raise at least one order of magnitude more if successful.

  2. 2

    Why it matters: Today's AI systems show occasional unpredictable failures, but humans can monitor and fix them. However, as AI systems become smarter and start handling more of their own development through recursive self-improvement, better alignment techniques will be critical to maintain control. Sequent argues that current frontier AI labs use methods that are "essentially reactive" and "do not yield principled insight into if or when they will fail." Independent organizations like Sequent may be better positioned to raise concerns if labs are taking unsafe approaches.

  3. 3

    What to watch: Sequent's research will focus on multiple areas—scalable oversight (methods to monitor AI behavior at scale), learning theory, game theory, and personas—with the belief that interactions between these directions could yield breakthroughs. The organization intends to pursue "a portfolio of differentiated alignment bets" rather than a single approach, in contrast to the strategies of major AI labs.

Discussion

No comments yet. Be the first to share your thoughts!

Log in to join the discussion

Related Articles

Stay ahead with AI news

Get curated AI news from 200+ sources delivered daily to your inbox. Free to use.

Get Started Free

Free · takes 30 seconds · unsubscribe anytime

5 minutes a day. The AI essentials.

200+ sources · Email / LINE / Slack

Get it free →