> ML_ARCHITECTURE // SEQ2SEQ-ENCODER-DECODER-TRANSFORMER_v1.0
Sequence-to-Sequence Encoder-Decoder Transformer (T5 / BART / Whisper)
Classic sequence-to-sequence architecture featuring a bidirectional encoder paired with an autoregressive decoder cross-attending to encoder representations, optimal for translation, summarization, and speech.
Transformerstextaudio
Back to All ArchitecturesArchitecture Overview
Classic sequence-to-sequence architecture featuring a bidirectional encoder paired with an autoregressive decoder cross-attending to encoder representations, optimal for translation, summarization, and speech.
Implementing Libraries
Seminal Papers
Attention Is All You NeedAshish Vaswani, Noam Shazeer (2017) · Advances in Neural Information Processing Systems (NeurIPS)
Exploring the Limits of Transfer Learning with a Unified Text-to-Text TransformerColin Raffel, Noam Shazeer (2020) · Journal of Machine Learning Research (JMLR)
Robust Speech Recognition via Large-Scale Weak Supervision (Whisper)Alec Radford, Jong Wook Kim (2022) · arXiv preprint
Architectural Limitations & Constraints
- Requires compatible deep learning framework and hardware acceleration for efficient execution.