ControlAI seeks $50M yearly to push for international ban on superintelligent AI development

ControlAI, an AI safety organization, announced it needs approximately $50 million per year in funding to pursue an international prohibition on the development of superintelligent AI (ASI — artificial systems that would vastly exceed human intelligence). The organization estimates this funding level would give them roughly a 10% chance of achieving this goal within the next few years.

Their strategy differs from technical AI safety research: rather than building safer AI systems, ControlAI focuses on persuading government decision-makers and the public to understand and act on extinction risks from superintelligent AI — essentially treating this as a policy and advocacy problem rather than an engineering one.

For business professionals and policymakers, this signals a growing belief among AI safety experts that superintelligent AI poses civilizational risks serious enough to warrant international legal restrictions — similar to nuclear weapons treaties. If this effort succeeds, it would constrain how private companies and governments can develop advanced AI systems in the future.