Skip to main content

> ML_LITERATURE // ROBBINS-MONRO-1951-STOCHASTIC-APPROXIMATION-METHOD_v1.0

A Stochastic Approximation Method

Herbert Robbins, Sutton Monro · The Annals of Mathematical Statistics (1951)

foundational1951industry-standardthirdPartyReproduced
DOI: 10.1214/aoms/1177729586Back to Literature

Principal Contribution

Introduced Stochastic Gradient Descent (SGD) and derived convergence conditions for learning rate schedules (sum eta_t = inf, sum eta_t^2 < inf).

Operational Relevance

Serves as canonical technical reference for implementing task-multiclass-classification, task-regression in production systems.

Assumptions

  • Underlying data distribution satisfies empirical consistency and regularity assumptions across training domains

Limitations

  • Scaling characteristics and accuracy depend on hardware architecture, parameter scale, and dataset quality

Connected Algorithms, Architectures & Tools

Related Algorithms:
alg-autoregressive-self-supervised-gpt
Related Architectures:
Implementing Libraries: