Data & Benchmarks

Zarr (Data Format)

Zarr is an open-source storage format for chunked, compressed N-dimensional arrays, designed for efficient parallel reads and writes on local disks and cloud object stores. Robot learning pipelines use it to store large observation-action datasets because random access to chunks supports shuffled training without loading whole files; the Diffusion Policy reference implementation, among others, distributes demonstration data as Zarr archives.

Why it matters for physical AI

Training generalist policies means streaming terabytes of multimodal trajectories into GPUs, and chunked array formats like Zarr are part of the data infrastructure that makes large-scale robot learning workloads practical.