Instruction-tuning

A Machine learning method1 to tune a generative model to follow human instructions. InstructGPT and ChatGPT are the most famous examples. Sebastian Raschka‘s Instruction Pretraining LLMs reviews some latest research (as of 202407).

Wang2022what investigates which language model architecture if better for Zero shot learning.

Wang2022self instruct suggests a method to use much less human resources.

  1. often via Reinforcement learning; see Training large language models with reinforcement learning ↩