Module 2: Language models and transformers · 10 min
Pre-training, fine-tuning and alignment
Building a chat assistant happens in stages.
| Stage | What happens | Result |
|---|---|---|
| Pre-training | The model reads a huge amount of text and learns to predict the next token | A base model that knows language and facts, but is not a good assistant |
| Fine-tuning | It is trained further on examples of good questions and answers | A model that follows instructions |
| Alignment | Humans (and other models) rate answers, and the model is trained to prefer helpful, honest and safe ones | A model that is more helpful and safer |
A model's knowledge stops at its training cut-off date. After that, it knows nothing new unless a tool such as web search or your own documents is connected to it.
Alignment reduces harmful and useless replies, but it does not make a model perfect.
← Attention and the transformer | How image models work: diffusion →