Nexus Learn

Log in · Create free account

Module 1: What is generative AI? · 10 min

How a model learns

A neural network is a large collection of numbers, called parameters or weights. At the start they are random. Training slowly adjusts them so the model makes fewer mistakes.

  1. Show the model an example from the training data.
  2. Let it make a guess (for a language model: guess the next word).
  3. Measure how wrong the guess was. This measure is called the loss.
  4. Nudge all the weights a tiny bit to make that guess less wrong.
  5. Repeat billions of times.

Big models have billions of parameters and need thousands of special chips (GPUs) running for weeks. This is why training a top model costs millions of dollars, while using one is cheap by comparison.

Three things decide how good a model is: the amount and quality of data, the size of the model, and the computing power spent on training.

(free account needed to save progress)

← AI, machine learning and generative AI  |  The main kinds of generative models →