Apple Shows Trillion-Parameter-Class LLMs Running Locally on Mac Studio Clusters with MLX
Apple's MLX team published benchmarks showing large open-weight models — including a near-trillion-parameter MoE model called Muse Spark Open — running on Mac Studio hardware, both on a single machine and distributed across up to four Mac Studios using a new RDMA-over-Thunderbolt driver. The new M5 Ultra chip, with dedicated GPU Neural Accelerators and 1.2 TB/s memory bandwidth, delivers up to 4.6x faster time-to-first-token and 1.6x faster generation speed versus the previous M3 Ultra. For models too large to fit on one machine, MLX now supports pipeline and tensor parallelization across multiple Macs connected via Thunderbolt RDMA, letting a four-Mac-Studio cluster generate over 36 tokens per second on an 816B-parameter model.