Google DeepMind's Pushmeet Kohli: AlphaFold didn't solve protein folding

The panel, moderated by Brandon Anderson, framed the discussion around the Bitter Lesson for biology. Sal Candido argued that scaling laws are not everywhere and always exist, and that a lot of the work is finding the right scaling law. He gave a concrete example: training a protein language model on metagenomic sequences, which he said are not the highest-quality data, improved performance for designing real proteins and understanding proteins that exist. Pushmeet Kohli recalled AlphaFold's development at DeepMind, where the team used existing datasets because it lacked the core expertise or resources to augment the PDB by a significant order of magnitude. He said the same approach applied to cell genomics made clear the data was not there yet to pursue building the virtual cell, which he described as a grand ambition. Kohli also described AlphaFold 2 as having a GDT score of about 90 on that set at the time, and argued that if its pLDDT score were completely uncalibrated, users would not trust it. He said he and John Jumper used to discuss what they were trying to solve, and that they did not know the actual true ground state proteins take or the actual distribution of structures they take. Candido added that current models solve a very specific purpose, using the analogy of modeling a spoke rather than a whole bicycle, and that moving from models of individual proteins to whole biological systems is where things are going. He also said protein language models contain information about functions and motions, not just structure, and that interpretability work has found a lot still to be unlocked.