Meta Launches Muse Spark Open, an 816B Multimodal Coding Model
Meta's Muse Spark Open combines 816 billion parameters, four input modes and a 1 million token window in an open-weight model optimized for coding agents.
Summary
Meta released Muse Spark Open on October 15, 2026, calling it the most capable US open-weight model to date. The coding-focused model accepts text, images, audio and video, outputs text, and supports agentic planning, tool use, codebase reading, debugging and self-correction. Its 62-layer mixture-of-experts architecture contains about 816 billion parameters, with 256 experts per layer, eight selected per token and about 42 billion active. Experts hold roughly 97% of parameters. The context limit is 1,048,576 tokens, although a full BF16 context requires about 266 GB for the KV cache. Audio produces 12.5 tokens per second, while video defaults to 2 FPS and 96 frames.
Meta provides Muse-Spark-Open-816B-A42B in BF16, a roughly 1.5 TB checkpoint running across 32 H200 GPUs, and Muse-Spark-Open-816B-A42B-NVFP4, a 4-bit Blackwell-optimized version with negligible stated accuracy loss that runs on one B200 node. Support spans Transformers, llama.cpp, SGLang, vLLM, oMLX, mlx-lm and Hugging Face Inference Endpoints. A 2-bit llama.cpp quant reached 18 tokens per second and used 304 GB on a 512 GB Mac M3 Ultra, while 3-bit MLX still needs more than 256 GB. LoRA fine-tuning on 64 H100 GPUs took about two hours for 500 steps at 8,000 tokens per second, lifting mean token accuracy from 72% to 84% and reducing loss from 1.26 to 0.57. Available demos cover iterative Three.js asset creation, screenshot-driven browser control and voice-guided Amazon searches.
Positives
- 816 billion parameters and four input modalities give developers one model for code, screenshots, recordings, voice notes and tool calls.
- Eight active experts per token reduce computation to roughly 42 billion parameters despite the model's 816 billion parameter capacity.
- The 1,048,576-token window supports long codebases and extended debugging workflows alongside image, audio and video tokens.
- NVFP4 cuts weights to 4-bit precision with negligible stated accuracy loss and fits inference on one B200 node.
- LoRA fine-tuning raised mean token accuracy from 72% to 84% after 500 steps while leaving all experts frozen.
- Support across Transformers, llama.cpp, SGLang, vLLM, MLX and Inference Endpoints broadens deployment options.
Risks & concerns
- The BF16 checkpoint occupies about 1.5 TB and requires 32 H200 GPUs in the documented Transformers configuration.
- All 256 experts must remain in memory even though only eight run per token, preserving the model's enormous memory requirement.
- A full one-million-token BF16 context needs about 266 GB solely for the KV cache.
- Even a 2-bit quant used 304 GB on a 512 GB Mac M3 Ultra, while 3-bit MLX requires more than 256 GB.
- Reasoning cannot be disabled, potentially increasing token consumption and latency on simple tasks.
- Video sampling trades accuracy against memory and prefill latency, with the default capped at 2 FPS and 96 frames.