Skip to main content

> ML_LITERATURE // HE-2016-DEEP-RESIDUAL-LEARNING-IMAGE-RECOGNITION_v1.0

Deep Residual Learning for Image Recognition

Kaiming He, Xiangyu Zhang, Shaoqing Ren, Jian Sun · IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2016)

seminal-architecture2016industry-standardthirdPartyReproduced
DOI: 10.1109/CVPR.2016.90Back to Literature

Principal Contribution

Introduced residual skip connections (y = F(x) + x), enabling training of networks with 100+ to 1000+ layers without degradation.

Operational Relevance

Residual connections are the ubiquitous architectural invariant present in ResNets, Transformers, Diffusion U-Nets, and Mamba.

Assumptions

  • Optimizing a residual mapping F(x) = H(x) - x is significantly easier than optimizing the original unreferenced mapping H(x)

Limitations

  • Pure convolution lacks global dynamic receptive field; eclipsed by Vision Transformers on massive data regimes

Connected Algorithms, Architectures & Tools

Related Algorithms:
Simple Framework for Contrastive Learning (SimCLR)alg-masked-autoencoder-mae
Related Architectures:
Implementing Libraries: