Robot Learning

Adam Optimizer

Adam is a stochastic gradient descent optimizer, introduced by Kingma and Ba in 2014, that adapts a separate learning rate for each parameter using exponential moving averages of the gradient and its square. Its robustness to hyperparameter choice made it the default optimizer for deep learning, and the decoupled weight decay variant AdamW is standard for training transformers, including robot policy networks and vision-language-action models.

Why it matters for physical AI

Virtually every robot policy trained today, from behavior-cloned manipulation networks to large vision-language-action models, is optimized with Adam or AdamW, so its convergence behavior quietly shapes the entire field's results.