Foundation Models

Byte-Pair Encoding (BPE)

Byte-pair encoding is a subword tokenization algorithm that builds a vocabulary by iteratively merging the most frequent pair of adjacent symbols in a corpus, adapted for neural machine translation by Sennrich et al. in 2016 and used by the GPT family. Beyond text, robotics has adopted BPE-style compression for action sequences: the FAST tokenizer applies it after a discrete cosine transform to encode high-frequency action chunks in few tokens.

Why it matters for physical AI

Compressed action vocabularies shorten the token sequences vision-language-action models must generate, directly cutting inference latency and making autoregressive policies practical for high-rate dexterous control.