Foundation Models

LoRA

LoRA (low-rank adaptation) is a parameter-efficient fine-tuning method, introduced by Hu et al. in 2021, that freezes a pretrained model's weights and injects small trainable low-rank matrices into selected layers, typically attention projections. Only the adapter parameters, often under one percent of the model, are updated, and they can be merged into the base weights at inference with no added latency.

Why it matters for physical AI

Adapting billion-parameter vision-language-action models to a new robot, gripper, or task with modest compute and data hinges on parameter-efficient methods, which also allow swapping lightweight task adapters on shared base weights.