Lecture 6.5 - RMSProp: Divide the gradient by a running average of its recent magnitude

Resolved AdaGrad diminishing learning rate problem by maintaining an exponentially decaying moving average of squared gradients.

Serves as qualified reference for implementing task-text-generation, task-image-classification in production systems.

Assumptions

• Underlying computational topology and mathematical bounds adhere to established convexity/smoothness guarantees

Limitations

• Hardware runtime speedups, privacy budgets, and convergence depend on hyperparameters and network communication limits