Robot Learning

Q-Learning

Q-learning is an off-policy reinforcement learning algorithm, introduced by Watkins in 1989, that learns the optimal action-value function through temporal-difference updates toward the observed reward plus the discounted maximum Q-value of the next state. In the tabular setting it converges to the optimal policy regardless of the exploration policy used. Deep Q-Networks (DQN) scaled the idea to high-dimensional inputs in 2015.

Why it matters for physical AI

Off-policy learning means data from any source, including old policies and human demonstrations, can improve the value function, a property robotics depends on given the cost of real-world experience.