Phase 043: Deep Learning Optimization
Phase 043 of the AI Encyclopedia — Deep Learning Optimization. Topics 0841–0860.
This phase covers Deep Learning Optimization. Below are the 20 concepts grouped under this phase — each is a future article in the Insightful AI World encyclopedia.
0841 SGD
0842 Mini-batch SGD
0843 Momentum
0844 Nesterov Momentum
0845 AdaGrad
0846 RMSProp
0847 Adam
0848 AdamW
0849 Learning Rate Scheduling
0850 Warmup
0851 Cosine Decay
0852 Gradient Clipping
0853 Weight Initialization
0854 Xavier Initialization
0855 He Initialization
0856 Vanishing Gradients
0857 Exploding Gradients
0858 Saddle Points
0859 Sharp vs Flat Minima
0860 Optimization Stability