Robot Learning

L2 Regularization

L2 regularization is a technique that penalizes large parameter values by adding the sum of squared weights, scaled by a coefficient, to a model's training loss. It encourages smaller, more evenly distributed weights, which reduces overfitting and improves generalization. In deep learning it is closely related to weight decay, though the two differ subtly under adaptive optimizers such as Adam, motivating the decoupled formulation used in AdamW.

Why it matters for physical AI

Robot policies are often trained on small, expensive demonstration datasets, so regularization is a primary defense against memorizing training trajectories instead of learning behaviors that generalize to new scenes and object placements.