Post-training
Everything done to a model after pre-training: instruction tuning, preference tuning, safety training and reasoning training.
Post-training turns a base model into a product. It usually starts with supervised fine-tuning on examples of good conversations, then preference tuning with RLHF or DPO, and increasingly reinforcement learning on tasks with checkable answers such as maths and code, which is how reasoning models are made. Safety behaviour, tool use and the assistant's character are shaped here too. Post-training uses far less compute than pre-training did in the early chatbot years, but labs have been spending a growing share on it since 2024.