> ML_LITERATURE // LIU-2021-SWIN-TRANSFORMER-HIERARCHICAL-VISION-TRANSFORMER-SHIFTED-WINDOWS_v1.0
Swin Transformer: Hierarchical Vision Transformer using Shifted Windows
Ze Liu, Yutong Lin, Yue Cao, Han Hu, Yixuan Wei, Zheng Zhang, Stephen Lin, Baining Guo · IEEE International Conference on Computer Vision (ICCV) (2021)
seminal-architecture2021industry-standardthirdPartyReproduced
Principal Contribution
Introduced shifted window self-attention, limiting attention computation to local non-overlapping windows with linear computational complexity.
Operational Relevance
ICCV Best Paper (Marr Prize) architecture serving as the premier dense prediction backbone for high-resolution detection and segmentation.
Assumptions
- Shifted window partitioning provides cross-window connections while preserving linear computational complexity with respect to image size
Limitations
- More complex implementation and CUDA kernel scheduling than standard global Vision Transformers