Wednesday, November 18, 2026
Tech Beat
Nov 18, 2026, 12:00 AMTechnology

180B Parameters, 32GB of RAM: Pushing MoE Inference to the Limit

Back to Articles a]:hidden"> 180B Parameters, 32GB of RAM: Pushing MoE Inference to the Limit with Transformers Published November 18, 2026 Update on GitHub…

Summary

Back to Articles a]:hidden"> 180B Parameters, 32GB of RAM: Pushing MoE Inference to the Limit with Transformers Published November 18, 2026 Update on GitHub Upvote 2 Tarek Ziadé tarekziade Follow:last-child]:mb-0"> experts_offload</code>","children":[],"isValid":true,"title":"<code>experts_offload</code>"},{"id":"running-121-gb-of-experts-on-a-32-gb-mac","label":"Running 121 GB of experts on a 32 GB Mac","children":[],"isValid":true,"title":"Running 121 GB of experts on a 32 GB Mac"}],"isValid":true,"title":"Making Qwen3. 8-Flash-Next work on my 32 GB Mac with Transformers"},{"id":"closing-the-gap","label":"Closing the gap","children":[],"isValid":true,"title":"Closing the gap"},{"id":"conclusion","label":"Conclusion","children":[],"isValid":true,"title":"Conclusion"}],"classNames":"not-prose hidden lg:block"}"> Streaming MoE experts directly from SSD gets within 6% of Strata on token generation. I've been trying to rely less on online models and run bigger ones locally. At home I have a PC with two RTX 4090s (let's call it Orthrus - the two-headed hound of Greek mythology), and for work I have a MacBook M5 with 32G of unified memory.

Positives

  • The report adds a new data point to the technology market.
  • The development may create new product, research, or competitive opportunities.

Risks & concerns

  • The article may not provide enough evidence to validate every implication.
  • Execution, cost, adoption, regulation, or security could change the outcome.
Primary sourceHugging Face - Bloghttps://huggingface.co/blog/moe-expert-streaming
Read full article
Editorial note: Tech Beat summarizes and analyzes third-party reporting. The source link is the authoritative article. This page does not reproduce the full source text.

More From The Wire

TechnologyNov 18

Why your fine-tuning job takes longer than the FLOPs say

TechnologyNov 18

Sony’s updated turntable is an easy entry into the world of vinyl

TechnologyNov 18

Inside the Codex sandbox