Phase 026: Optimization for ML
Phase 026 of the AI Encyclopedia — Optimization for ML. Topics 0501–0520.
This phase covers Optimization for ML. Below are the 20 concepts grouped under this phase — each is a future article in the Insightful AI World encyclopedia.
0501 Optimization Problem Formulation
0502 Convex Optimization
0503 Non-convex Optimization
0504 Gradient Descent
0505 Stochastic Gradient Descent
0506 Mini-batch Gradient Descent
0507 Momentum
0508 Nesterov Momentum
0509 AdaGrad
0510 RMSProp
0511 Adam
0512 AdamW
0513 Learning Rate Scheduling
0514 Line Search
0515 Constraints
0516 Lagrange Multipliers
0517 KKT Conditions
0518 Regularized Optimization
0519 Numerical Optimization
0520 Optimization Pathologies