Robot Learning

Mixed Precision Training

Mixed precision training is a deep learning technique that performs most computation in reduced-precision formats such as FP16 or BF16 while keeping selected accumulations and a master copy of weights in FP32, roughly doubling throughput and halving memory on modern accelerators. FP16 training uses loss scaling to prevent gradient underflow, whereas BF16's wider dynamic range typically avoids the need, making it the default on recent hardware.

Why it matters for physical AI

Training billion-parameter vision-language-action models on massive demonstration corpora is compute-bound; precision engineering substantially cuts cost and memory, and similar quantization thinking extends to fast on-robot inference.