Nexus Learn

Log in · Create free account

Module 2: Language models and transformers · 10 min

Pre-training, fine-tuning and alignment

Building a chat assistant happens in stages.

StageWhat happensResult
Pre-trainingThe model reads a huge amount of text and learns to predict the next tokenA base model that knows language and facts, but is not a good assistant
Fine-tuningIt is trained further on examples of good questions and answersA model that follows instructions
AlignmentHumans (and other models) rate answers, and the model is trained to prefer helpful, honest and safe onesA model that is more helpful and safer

A model's knowledge stops at its training cut-off date. After that, it knows nothing new unless a tool such as web search or your own documents is connected to it.

Alignment reduces harmful and useless replies, but it does not make a model perfect.

(free account needed to save progress)

← Attention and the transformer  |  How image models work: diffusion →