QLoRA
LoRA fine-tuning on top of a 4-bit quantised model, which let large models be tuned on a single GPU.
QLoRA, published by Tim Dettmers and colleagues at the University of Washington in 2023, combines a frozen 4-bit quantised base model with trainable LoRA adapters kept in higher precision. With it, a 65-billion-parameter model could be fine-tuned on one 48-gigabyte graphics card. The paper introduced a 4-bit number format suited to the bell-shaped spread of neural network weights. QLoRA made fine-tuning large open models practical for universities, hobbyists and small companies.