Data & Benchmarks

Ego4D

Ego4D is a large-scale egocentric video dataset and benchmark suite released in 2022 by Meta AI and a consortium of universities, containing roughly 3,670 hours of daily-life first-person video from over 900 camera wearers across many countries. It includes annotations for episodic memory, hand-object interaction, forecasting, and social understanding tasks. Ego4D has become a standard pretraining corpus for egocentric perception and video representation models.

Why it matters for physical AI

First-person human video is the closest abundant proxy for robot manipulation viewpoints, and representations pretrained on Ego4D, such as R3M, transfer measurably to robot policy learning.