Adapting giant models grew affordable through a small mathematical trick. A 2021 method freezes a pretrained model and trains only tiny low-rank matrices injected into its layers. It cut the cost of customization by orders of magnitude.
Freeze and inject
The base stays fixed. Only compact adapter matrices are trained, capturing the task-specific change. The original weights are untouched.
Tiny footprint
Storage shrinks. An adapter is a fraction of a percent of the full model. Many tasks share one base.
No inference penalty
Speed is preserved. The low-rank update can be merged back at deployment. Latency is unchanged.
Democratized tuning
Access broadened. Researchers with modest hardware could specialize huge models. Experimentation accelerated.
Composability
Adapters combine. Different skills can be swapped or stacked at run time. Modularity emerged.
Wide reach
Use spread fast. The method became standard for adapting language and image models alike. It anchors efficient fine-tuning.
The bottom line
Low-rank adaptation trains a tiny fraction of parameters to specialize huge models. It preserves speed and slashes cost. The method is now the default for efficient fine-tuning.
