Robot Learning

Replay Buffer

A replay buffer is a memory that stores past interaction transitions, typically tuples of state, action, reward, and next state, from which off-policy reinforcement learning algorithms sample minibatches for updates. Introduced as experience replay and popularized by DQN, it decorrelates sequential samples and reuses experience many times, with prioritized replay weighting samples by learning progress. Its logic extends naturally to offline RL on fixed datasets.

Why it matters for physical AI

Sample reuse is precious when experience comes from physical hardware, and replay-based off-policy methods are the standard choice for the modest data budgets of real-robot reinforcement learning.