Foundation Models

Vision-Language Model (VLM)

A vision-language model (VLM) is a neural network trained on paired image and text data to jointly represent and reason over visual and linguistic content, supporting tasks such as captioning, visual question answering, and open-vocabulary grounding. Contrastive models like CLIP align images and text in a shared embedding space, while generative VLMs such as PaLI, LLaVA, and GPT-4-class multimodal models produce free-form text conditioned on images.

Why it matters for physical AI

VLMs give robots open-vocabulary perception and commonsense scene understanding for free, and they are the pretrained substrate from which vision-language-action models are fine-tuned on robot data.