PyTorch 2.15 brings FP8 to Mac MPS, 61.3 tokens/s on M3 Ultra

The PyTorch team at Hugging Face describes a deliberate push to make Apple's MPS backend a first-class place to run and train models. At the start of 2026, the team says, running even small models on MPS often meant working around unsupported operations, performance cliffs and unexpected memory growth. Their method is to run real models from the Hugging Face Hub across language, image and video, find where MPS falls short, and fix it upstream in PyTorch — increasingly by replacing MPSGraph with dedicated Metal kernels.

That approach shows up across the 2.15 release. FP8 arrives on MPS alongside scaled matrix multiplication, and separate work targets the decode phase of language models: linear layers get specialized matrix-vector kernels, and softmax over large vocabularies runs on dedicated Metal kernels. The team reports an 8× faster FP32 softmax over a [1, 128256] row of logits. Beyond language, constant padding moving to Metal fixes runaway driver-memory growth in tiled video VAE decoding, and an encode-decode round trip for Qwen-Image-2.1 now matches CPU at 29.95 dB PSNR, up from 10.06 dB.

The post also sketches what comes next. PyTorch 2.16 is already underway, with scaled dot-product attention dropping MPSGraph entirely so every shape runs on Metal kernels, and more MPSGraph-to-Metal migrations in progress. The team also says it is working on torch.compile improvements to eliminate graph breaks so more models compile end to end.