Meta releases 816B-parameter Muse Spark Open model

Meta released Muse Spark Open on Oct. 15, with BF16 and NVFP4 weights for the 816-billion-parameter multimodal model now available to download. Developers can run the weights on their own infrastructure, although Meta says even the lower-precision NVFP4 version needs at least 700GB of GPU memory. The release is separate from Meta’s Muse Wonder model launch earlier in the week.

Meta describes Muse Spark Open as a mixture-of-experts Transformer with 42 billion active parameters. For each token, the model routes work through a subset of its components: Meta says it selects eight of 256 language-model experts. That cuts the computation used per token, but it does not eliminate the need to keep the full 816-billion-parameter weight set in memory.

The model has 62 layers, an 8,192-wide hidden dimension and a perception encoder, according to Meta. The company says it accepts text, images, video and audio, produces text, and supports an inference context of up to 1 million tokens. The repository includes BF16 and lower-precision NVFP4 weights. Meta puts the minimum self-hosting memory at 2TB for BF16 and 700GB for NVFP4. Those capacity figures are Meta’s stated requirements, not independent measurements.

Apple’s same-day MLX tests show what that footprint looks like on desktop hardware. Apple said the NVFP4 model would not fit on a single Mac Studio with 512GB of unified memory, so its baseline split the model across two M5 Ultra Mac Studios using pipeline parallelism. The test reflects the difference between active parameters and total storage: activating 42 billion parameters for a token lowers computation without reducing the entire weight set to 42 billion parameters.

Apple said it used mlx_lm.benchmark with an 8,192-token context and measured decoding over 128 generated tokens. It reported 15.74 tokens per second on two M5 Ultra systems with pipeline parallelism, 24.99 tokens per second on two systems with tensor parallelism, and 36.73 tokens per second on four systems with tensor parallelism. These are Apple’s measurements and have not been independently reproduced.