Decoupled Weight Decay Regularization (AdamW)

Decoupled L2 regularization from gradient updates in Adam, proving that standard L2 penalty fails in adaptive optimizers and restoring true weight decay.

Serves as qualified theoretical and empirical reference for task-text-generation, task-image-classification.

Assumptions

• Mathematical convexity, regularity, and empirical consistency hold across problem configurations

Limitations

• Theoretical bounds and benchmark saturation characteristics vary across model architectures and training scales