Robot Learning

Belief State

A belief state is a probability distribution over the possible true states of a partially observable system, summarizing everything the observation history implies about the world. It is the sufficient statistic on which optimal policies for partially observable Markov decision processes (POMDPs) operate, and it is maintained in practice by Bayes filters such as Kalman and particle filters, or implicitly by the hidden state of recurrent and transformer policies.

Why it matters for physical AI

Robots never observe the world fully, with occlusion, sensor noise, and unknown object properties everywhere, so acting on calibrated uncertainty rather than point estimates separates robust deployed behavior from brittle demos.