Data & Benchmarks

Episode Format

Episode Format is the on-disk schema used to store robot interaction episodes, specifying how time-aligned streams of camera frames, proprioception, actions, language annotations, and metadata are serialized. Common choices include RLDS built on TFRecords, used by Open X-Embodiment, HDF5 layouts popularized by robomimic, and the LeRobot format, which stores video-compressed frames with tabular data. Formats trade random-access speed, compression, and streaming behavior against tooling compatibility.

Why it matters for physical AI

Format fragmentation is a real tax on the field: pooling datasets across labs and training on mixtures requires costly conversion, so converging on standard episode schemas accelerates every downstream effort.