> ML_LITERATURE // DAO-2024-TRANSFORMERS-ARE-SSMS-MAMBA2_v1.0
Transformers are SSMs: Generalized Models and Efficient Algorithms Through Structured State Space Duality
Tri Dao, Albert Gu · International Conference on Machine Learning (ICML) (2024)
seminal-architecture2024industry-standardthirdPartyReproduced
Principal Contribution
Established Structured State Space Duality (SSD), proving formal theoretical equivalence between selective state-space models and masked linear attention, unlocking Tensor Core matrix multiplication.
Operational Relevance
Directly guides architectural decisions, alignment strategy, and serving infrastructure for task-text-generation.
Assumptions
- Empirical distribution regularity holds and target domain adheres to pretraining linguistic/visual support
Limitations
- Resource scaling, inference memory requirements, and alignment robustness vary with model size and hardware topology