Robot Learning

Hindsight Experience Replay (HER)

Hindsight Experience Replay is a technique for goal-conditioned reinforcement learning, introduced by Andrychowicz et al. (2017), that relabels failed trajectories as successes for the goals the agent actually reached. By storing each transition with substituted achieved goals, HER extracts dense learning signal from sparse binary rewards, and it enabled off-policy algorithms like DDPG to solve robotic pushing, sliding, and pick-and-place benchmarks.

Why it matters for physical AI

Reward engineering is a major bottleneck for robot RL; hindsight relabeling turns every attempt into useful data, improving sample efficiency where real-robot experience is expensive and rewards are naturally sparse.