Foundation Models

Visual Foundation Model

A visual foundation model is a large vision network pretrained on web-scale image data whose representations transfer broadly across downstream tasks without task-specific training. Prominent examples include DINOv2 for self-supervised features, SAM (Segment Anything) for promptable segmentation, and CLIP's image tower for language-aligned embeddings. Robotics pipelines use them frozen or lightly fine-tuned for detection, segmentation, correspondence, and as policy encoders.

Why it matters for physical AI

Robot datasets are small compared to internet imagery, so leaning on visual foundation models for perception lets manipulation and navigation systems generalize to novel objects and scenes their training data never covered.