Robot Learning

Hierarchical Reinforcement Learning (HRL)

Hierarchical reinforcement learning is a family of RL methods that decompose long-horizon problems using temporal abstraction, learning high-level policies that invoke temporally extended lower-level behaviors. The options framework of Sutton, Precup, and Singh formalized skills as options with initiation sets and termination conditions; later approaches include feudal networks and goal-conditioned hierarchies such as HIRO.

Why it matters for physical AI

Sparse rewards over hundreds of steps cripple flat RL; temporal abstraction shortens the effective horizon, improving exploration and credit assignment for multi-stage manipulation and locomotion tasks.