Google releases TorchTPU to run PyTorch apps on its TPUs

PyTorch was originally developed by Meta in 2016, long before LLMs changed the world, and many of its features were built to work specifically with Nvidia's GPUs, which were already the standard hardware for AI research at the time. In 2022, Meta turned PyTorch over to the Linux Foundation. Because most AI apps are written by default to run on Nvidia's chips, Google's release of TorchTPU — and Amazon's release of TorchNeuron on Monday — represent efforts to offer alternative paths. Google says it worked directly with Meta, Hugging Face, Inferact (maker of vLLM), RadixArk (maker of SGLang), and others to ensure that TorchTPU's features and code libraries work well with these popular AI development tools, just as the original PyTorch does. Google also launched a dedicated tool, the TPU agent suite, to help developers move their apps to Google's chips and cloud and fix errors along the way, and updated its LiteRT tool, which runs AI on phones, laptops and embedded devices, to work better with PyTorch AI apps. None of this means AI app developers will automatically ditch Nvidia's hardware en masse just because they can now easily move to alternative chips and clouds, but Google and Amazon are clearly working to end Nvidia's advantage.