LoRA (Low-Rank Adaptation)

LLMs

A method to fine-tune LLMs without performing the full Finetuning (Hu2021LoRA). It’s one of the most widely used approach.

Practical Tips for Finetuning LLMs Using LoRA by Sebastian Raschka

Dettmers2023QLoRA