Edge0 Releases 35B MoE Model That Runs in 3 GB of Memory

Edge0-35B-A3B-preview has been published on HuggingFace — a 35B-class text model with a sparse MoE architecture. The authors claim it decodes at 15 tokens/s while actively using less than 3 GB of memory.

What it means

The model has 34.7 billion parameters, the weights in the repository take up 18.3 GB in safetensors format, and the license is apache-2.0. It runs through the edge0 framework with streaming expert loading: the full 4-bit checkpoint stays on disk, and only the active weights are loaded into memory. There are no quantized builds on the site yet — the checkpoint already ships in int4 along with LoRA and prerouter adapters. Running it requires the edge0 framework itself, which streams expert weights from storage on demand. Peak memory is limited by the active set of weights rather than the total number of parameters, so the model fits in a phone's memory without sharding. How sensitive this approach is to disk speed, the developer does not specify. The model is in early preview status, and it is too early to judge its practical applicability. But the approach itself — keeping weights on disk and loading only the experts you need — is interesting for those running LLMs on machines with limited memory. You can follow its development in the CPU3D news feed.