Oh wow, Mamba-3 is here!
For me, the most interesting use case of Mamba and Mamba-likes are the recent transformer attention hybrid architectures (Qwen3.5, Kimi Linear, etc.)
Would be interesting to swap Gated DeltaNet with Mamba-3 (which now also has RoPE) in next gen hybrids.
#### Albert Gu
@_albertgu · 13h ago
The newest model in the Mamba series is finally here 🐍 Hybrid models have become increasingly popular, raising the importance of designing the next generation of linear models.
We've introduced several SSM-centric ideas to significantly increase Mamba-2's modeling capabilities without compromising on speed. The resulting Mamba-3 model has noticeable performance gains over the most popular previous linear models (such as Mamba-2 and Gated DeltaNet) at all sizes.
This is the first Mamba that was student led: all credit to @aakash_lahoti @kevinyli_ @_berlinchen @caitWW9, and of course @tri_dao!
24
209
1,042
206.8K
7 Replies
27 Retweets
238 Likes
12.7K Views 
One Sentence Summary
Sebastian Raschka highlights the release of Mamba-3, emphasizing its potential to enhance transformer attention hybrid architectures like Qwen3.5 and Kimi Linear, especially with the inclusion of RoPE.
Summary
This tweet by Sebastian Raschka announces the release of Mamba-3, a significant update in the Mamba series of linear models. Raschka specifically points out the most interesting application of Mamba-3 in recent transformer attention hybrid architectures such as Qwen3.5 and Kimi Linear. He suggests that Mamba-3, now incorporating RoPE (Rotary Positional Embeddings), could be a strong candidate to replace Gated DeltaNet in next-generation hybrid models, indicating potential architectural improvements. The quoted tweet from _albertgu, one of Mamba's creators, confirms Mamba-3's arrival, detailing its SSM-centric improvements over Mamba-2 and Gated DeltaNet, and noting its performance gains across all sizes. This release underscores the growing importance of linear models in hybrid AI designs.
AI Score
85
Influence Score 40
Published At Today
Language
English
Tags
Mamba-3
SSM
Hybrid Architectures
Transformer
RoPE