Foundation Models

Pre-Training

Pre-training is the initial training of a model on a large, broad dataset to acquire general representations before fine-tuning on a downstream task. In robotics this takes several forms: visual encoders pretrained on human video as in R3M and VC-1, cross-embodiment pre-training on pooled robot datasets such as Open X-Embodiment, and initializing vision-language-action models from web-pretrained vision-language backbones.

Why it matters for physical AI

Robot data is scarce and expensive, so leveraging knowledge pretrained on web and cross-robot corpora is the central strategy by which robot foundation models escape the limits of per-task data collection.