Trillium Labs to open up risky AI research, aims for $40 to $100 million

Trillium Labs arrives as the industry is split over whether powerful models should stay locked inside a few labs or be opened for outside scrutiny. Lambert, who previously worked at the open-research lab Ai2 and at Hugging Face, argues that the closed path of frontier AI development takes the field backwards. He and Zick met during the COVID-19 pandemic while both were graduate students at UC Berkeley, and they got the idea after seeing how disconnected industry AI research has become from academic work.

The nonprofit plans to publish details of experiments so outside scientists can study and replicate them, starting with post-training and recursive self-improvement. The topic gained mainstream attention earlier this month when an Anthropic researcher left the company and warned that recursive self-improvement could pose an existential threat to humankind. Trillium Labs says it has raised an undisclosed sum from Schmidt Sciences, Halcyon Futures, and others, and aims to raise $40 to $100 million in total.

Whether the open approach produces useful scrutiny may hinge on whether the lab can fund the compute its founders say careful experimentation requires. Zick notes that understanding how reinforcement learning scales in post-training needs significant compute and many careful experiments, so the $30 million planned for training over 18 months could be a test of how far transparency can stretch on a nonprofit budget.