Robot Learning

Proximal Policy Optimization (PPO)

Proximal policy optimization (PPO) is a policy-gradient reinforcement learning algorithm introduced by Schulman et al. in 2017 that stabilizes training by clipping the probability ratio between the updated and previous policy, approximating a trust-region constraint with a simple first-order objective. Its robustness to hyperparameters and easy parallelization made it the default algorithm for large-scale simulated robot learning.

Why it matters for physical AI

PPO paired with massively parallel GPU simulation is the workhorse recipe behind most modern sim-to-real locomotion results, from quadruped parkour to humanoid whole-body control.