Google unveils TorchTPU to run PyTorch AI on TPUs

TorchTPU arrives as an alternative to the PyTorch framework that Meta created in 2016, before large language models drove the current AI boom. Because PyTorch was designed when Nvidia's GPUs were already the preferred hardware for AI research, most AI applications have continued to target that hardware even after Meta open-sourced PyTorch in 2022 and handed stewardship to the Linux Foundation.

Google's release follows Amazon Web Services' announcement earlier this week of TorchNeuron, its own flavor of PyTorch for Amazon's Trainum chips. Amazon also contributed code to the original PyTorch project to decouple key features from Nvidia's GPUs. Google did not do that, but it worked with Meta, Hugging Face, Inferact Inc. and RadixArk Inc. on compatibility with their developer tools, which may give it an edge over AWS in that respect.

Beyond TorchTPU, Google revealed a TPU agent suite to help developers move AI applications to Google Cloud and automatically fix errors during migration. It also updated LiteRT, a tool for building AI applications that run directly on smartphones, laptops and embedded devices, so it should now be compatible with PyTorch-based apps. The company still faces the challenge of convincing millions of developers to shift away from their current hardware, but the release suggests Google is confident it can chip away at Nvidia's lead in the longer term.