> ML_LITERATURE // GLOROT-BENGIO-2010-UNDERSTANDING-DIFFICULTY-TRAINING-DEEP-FEEDFORWARD-NEURAL-NETWORKS_v1.0
Understanding the difficulty of training deep feedforward neural networks (Xavier / Glorot Initialization)
Xavier Glorot, Yoshua Bengio · International Conference on Artificial Intelligence and Statistics (AISTATS) (2010)
foundational2010industry-standardthirdPartyReproduced
Principal Contribution
Analyzed activation and gradient variances across deep layers; proposed Xavier/Glorot initialization: Var(W) = 2 / (fan_in + fan_out).
Operational Relevance
Serves as canonical technical reference for implementing task-multiclass-classification in production systems.
Assumptions
- Underlying data distribution satisfies empirical consistency and regularity assumptions across training domains
Limitations
- Scaling characteristics and accuracy depend on hardware architecture, parameter scale, and dataset quality
Connected Algorithms, Architectures & Tools
Related Algorithms:
Related Architectures:
Implementing Libraries: