PyTorch 2.15 Brings FP8 and Major MPS Performance Gains to Apple Silicon
PyTorch 2.15 brings FP8 to Apple silicon, speeds Mac LLM decoding, cuts Wan2.2 VAE memory from 270 GiB to 8 GiB, and expands Metal across AI workloads.
Summary
PyTorch 2.15’s release candidate, detailed October 20, 2026 by Hugging Face’s PyTorch team, strengthens local AI on Apple silicon but requires macOS 15 and Python 3.11. MPS gains one-byte torch.float8_e4m3fn tensors, half FP16 or BF16 storage, plus tensorwise and rowwise F.scaled_mm on the GPU. On an M3 Ultra with torch.compile, Qwen3.6-35B-A3B rises from 9.4 to 61.3 tokens per second and LFM2.5-8B-A1B from 23.6 to 123.6. MoE decoding is 5 to 6.5 times faster than in 2.14 through native grouped matrix multiplication, fewer GPU and CPU synchronizations and faster top-k routing. GEMV-backed F.linear averages 1.23 times faster across 96 BF16 shapes, while FP32 softmax over 128,256 logits falls from 57 to 7 microseconds, roughly eightfold.
Metal padding cuts Wan2.2’s 8,764-call, 28-tile VAE decode from 270.135 GiB to 8.108 GiB peak driver memory. The same change fixes Qwen-Image-2.1 corruption, lifting CPU-matched encode-decode quality from 10.06 to 29.95 dB PSNR. New Metal paths cover convolutions, reflection and replication padding, dropout, upsampling backward, channels-last conv2d backward and reductions. MPS also gains complex64 LU and triangular solves, faster inverses and QR, more robust SVD, Python streams, improved event timing, additional operations and corrected attention broadcasting. PyTorch 2.16 targets all-Metal attention, more MPSGraph replacements, 4% to 19% faster compiled decoding and a roughly 25% smaller libtorch_cpu.dylib, although preview details may change.
Positives
- Qwen3.6-35B-A3B reaches 61.3 tokens per second on an M3 Ultra, up from 9.4 in PyTorch 2.14.
- Wan2.2 VAE decoding peaks at 8.108 GiB of driver memory instead of 270.135 GiB after padding moves to Metal.
- FP8 MPS tensors use one byte per value, halving the storage required by FP16 or BF16.
- Qwen-Image-2.1 encode-decode quality improves from 10.06 to 29.95 dB PSNR, restoring image-editing correctness.
- A 128,256-element FP32 softmax completes in 7 microseconds instead of 57 microseconds.
- Python developers gain public torch.mps.Stream controls and allocator-aware Tensor.record_stream support.
Risks & concerns
- macOS 15 or later is mandatory, excluding Apple silicon Macs that cannot upgrade.
- Python 3.11 or later is required, potentially forcing environment migrations for existing projects.
- torch.logspace still lacks multi-element integer and Boolean output support on MPS.
- Large linear layers and language-model heads remain near their previous performance because they were already bandwidth-bound.
- PyTorch 2.16 features remain partly in review, so its all-Metal attention and other preview details may change.
