MLX LM v0.32.2 loads Meta's Muse Spark from Hugging Face and fixes MiMo-V2-Flash
MLX LM now loads Meta's Muse Spark from the checkpoint Meta put on Hugging Face. The earlier support expected the weight names of an internal checkpoint, and every name now matches the public release.
Muse Spark's output layer is half the size
Muse Spark’s output layer is half the size
The public Muse Spark checkpoint stores its output layer in 16-bit precision, where the internal one used 32-bit. That layer runs on every token, so decoding should get a bit faster.
MiMo-V2-Flash now scales its attention values
MiMo-V2-Flash's config tells it to scale its attention values by about 0.7 before they enter the cache. MLX LM skipped that step, so the model drifted from its reference code. Now it matches.
Padding stops fading the state of Mamba-2 and other state space models
Padding stops fading the state of state space models
In a padded batch, state space models let the padding tokens decay the state carried over from the cache. Mamba-2, Nemotron-H, Falcon-H1, Granite's hybrids and PLaMo 2 now mask the padding out of that decay.
