Morning Edition · Tuesday, August 4, 2026Published at 2:16 AM EDT · New York

An Open-Source Runtime Streams Mixture-of-Experts Weights From SSD to Run an 80-Billion-Parameter Qwen on a Mac

Swiftlet keeps only the dense core in memory and fetches each routed expert with a single disk read, putting a 35-billion-parameter model on an iPhone at roughly one token per second.

A Swift and Metal runtime called Swiftlet applies a straightforward observation about sparse models to deployment. In a mixture-of-experts (MoE) model, only a small fraction of parameters activate per token, so the routed experts do not nee…