Perception

Dense Optical Flow

Dense Optical Flow is the per-pixel estimation of apparent motion between consecutive images, producing a vector field over the entire frame rather than at sparse feature points. Classical variational methods such as Horn-Schunck have largely been superseded by learned estimators, with RAFT (2020) establishing the modern recurrent refinement paradigm. Flow supports motion segmentation, visual odometry, and video-based dynamics understanding.

Why it matters for physical AI

Motion cues let robots segment moving objects, estimate ego-motion, and track manipulated items without task-specific labels. Flow-based representations also serve as intermediate action abstractions in some manipulation policies trained from video.