Tutorial on LLM Post-Training Explains How Models Learn to Converse and Reason

A primer on post-training for large language models was originally written for the Meta infrastructure team and is now being shared broadly, targeting infrastructure engineers without LLM modeling expertise who want to learn how post-training enables capabilities like reasoning, tool use, and code generation.

Post-training (also called alignment) teaches models to follow conversational rules—such as taking turns and listening before responding—by using techniques including Supervised Fine Tuning (SFT), where models learn to imitate ideal responses word-by-word, and Rejection Sampling, where the model generates its own training responses from multiple checkpoints and seeds rather than relying solely on human-written answers.

The post-training phase operates on far smaller data scales than pre-training, using only a few million samples and a few billion tokens rather than trillions, and works by masking the loss function to condition on system and user prompts without learning from them directly.