Meta releases 816B-parameter open-weight Muse Spark Open

Muse Spark Open extends Meta's Muse series beyond Muse Glimmer, which Meta released in August as a small dense model from the same series. Both models share the same tokenizer and vocabulary, and the chat template follows Muse Glimmer's, with additions for the new modalities. The architectural change is the swap of Muse Glimmer's single dense feed-forward network per layer for a Mixture-of-Experts block with 256 experts, of which a router selects 8 per token. Those experts hold about 97% of the model's parameters, which is why the deployment footprint matches an 816B-parameter model while per-token computation matches a 42B one.

The multimodal side is handled by separate encoders. A 1.8B-parameter perception encoder turns images and video frames into tokens, and an audio encoder emits one token every 80 milliseconds. All of these tokens share the 1M-token context window with text, and grouped-query attention keeps the KV cache to about 250 KB per token in BF16, or about 266 GB for a full 1M-token context.

Because the full-weight model is heavy, several serving paths are offered to make it practical. Quantized versions are integrated in llama.cpp, and the NVFP4 checkpoint is supported in SGLang and vLLM, with an SGLang recipe available for Hugging Face Inference Endpoints. Meta trained the model specifically for agentic coding tasks such as multi-file refactors, long debugging sessions, and tool calling.