Robot Learning

Value Function

A value function is an estimate of the expected cumulative reward an agent will accumulate from a given state, written V(s), or from a state-action pair, written Q(s, a), while following a particular policy. Value functions satisfy the Bellman equation and are the central object of dynamic programming, temporal-difference learning, and actor-critic reinforcement learning, where a learned critic evaluates actions proposed by the policy.

Why it matters for physical AI

Value estimates let robots trained with reinforcement learning trade off long-horizon consequences of actions, and pretrained value functions are increasingly used to score, filter, or steer imitation-learned policies at deployment time.