180B Parameters, 32GB of RAM: Pushing MoE Inference to the Limit
Back to Articles a]:hidden"> 180B Parameters, 32GB of RAM: Pushing MoE Inference to the Limit with Transformers Published November 18, 2026 Update on GitHub…
Summary
Back to Articles a]:hidden"> 180B Parameters, 32GB of RAM: Pushing MoE Inference to the Limit with Transformers Published November 18, 2026 Update on GitHub Upvote 2 Tarek Ziadé tarekziade Follow:last-child]:mb-0"> experts_offload</code>","children":[],"isValid":true,"title":"<code>experts_offload</code>"},{"id":"running-121-gb-of-experts-on-a-32-gb-mac","label":"Running 121 GB of experts on a 32 GB Mac","children":[],"isValid":true,"title":"Running 121 GB of experts on a 32 GB Mac"}],"isValid":true,"title":"Making Qwen3. 8-Flash-Next work on my 32 GB Mac with Transformers"},{"id":"closing-the-gap","label":"Closing the gap","children":[],"isValid":true,"title":"Closing the gap"},{"id":"conclusion","label":"Conclusion","children":[],"isValid":true,"title":"Conclusion"}],"classNames":"not-prose hidden lg:block"}"> Streaming MoE experts directly from SSD gets within 6% of Strata on token generation. I've been trying to rely less on online models and run bigger ones locally. At home I have a PC with two RTX 4090s (let's call it Orthrus - the two-headed hound of Greek mythology), and for work I have a MacBook M5 with 32G of unified memory.
Positives
- The report adds a new data point to the technology market.
- The development may create new product, research, or competitive opportunities.
Risks & concerns
- The article may not provide enough evidence to validate every implication.
- Execution, cost, adoption, regulation, or security could change the outcome.