Robot Learning

Latent Space

A latent space is a learned, typically lower-dimensional vector space in which a model represents data, with coordinates capturing abstract factors of variation rather than raw measurements. Encoders such as VAEs, contrastive models, and autoencoding transformers map images, states, or trajectories into latent vectors that support downstream prediction, planning, or control. Distances and directions in a well-structured latent space often correspond to semantically meaningful changes.

Why it matters for physical AI

Planning and control in latent space, as in world-model agents like Dreamer, sidesteps raw-pixel dynamics and lets robots reason compactly about high-dimensional camera observations in real time.