Nexus Learn

Lecture 24: Machine Learning: Supervised, Unsupervised and Reinforcement

Lecture Slides CS-511 Artificial Intelligence 6 min read Updated 8 Nov 2026

↓ Download slides (PowerPoint)

Recap of Lecture 23 and Homework Check

LAST LECTURE · 5 MIN

Today: the fan rules came from an expert. Could a computer LEARN good rules by itself - from examples, or by trial and error?

Today's Plan (50 minutes)

ROADMAP

0-5

Recap of Lecture 23 and homework check

S02

5-8

Plan and learning outcomes

S03-S04

8-18

Module 1: What is machine learning? Three kinds of learning

S05-S07

18-27

Module 2: Supervised and unsupervised learning

S08-S11

27-35

Module 3: Reinforcement learning and Q-learning

S12-S15

35-43

Class activity: which kind of learning?

S16

43-48

Common mistakes and recap quiz

S17-S18

48-50

Homework and next lecture

S19

Learning Outcomes (SLOs)

BY THE END OF THIS LECTURE YOU WILL BE ABLE TO

SLO 24.1

Explain machine learning and differentiate supervised, unsupervised and reinforcement learning with examples

→ CLO3 · C2 Understand

SLO 24.2

Apply k-nearest neighbours to labelled data and one k-means step to unlabelled data

→ CLO3 · C3 Apply

SLO 24.3

Apply the Q-learning update rule to a small reinforcement learning problem

→ CLO3 · C3 Apply

  • Course Learning Outcome(s) served
  • CLO3: Apply various techniques and algorithms of AI used in general problem solving, optimization problems, constraint satisfaction problems, and game programming.

Slide 5

MODULE 1

What Is Machine Learning?

Learning the rules from data

Traditional Programming vs Machine Learning

MODULE 1 · THE IDEA

TWO WAYS TO GET A PROGRAM

  • Traditional: rules + data -> answers
  • Machine learning: data + answers -> rules (a model)
  • model + new data -> answers

Learning means: performance on the task gets better with experience - not 'the computer becomes intelligent'.

Three Kinds of Learning

MODULE 1 · THE BIG MAP

Supervised

Unsupervised

Reinforcement

  • Examples WITH answers (labels).
  • Like practising with a solved past paper.
  • Predict pass/fail, wheat yield.
  • Examples WITHOUT answers.
  • Find groups or patterns by itself.
  • Group customers by spending.
  • Actions and REWARDS.
  • Learn by trial and error.
  • A robot learns to walk; AlphaGo.

Slide 8

MODULE 2

Supervised and Unsupervised Learning

With answers and without

Supervised Learning: Learning from Labelled Examples

MODULE 2 · SUPERVISED

REAL OUTPUT: L24/SUPERVISED.PY (K-NN, K = 3)

  • trained on 8 students, tested on 4
  • 9 h, 75 % true pass predicted pass
  • 6 h, 58 % true fail predicted fail
  • 10 h, 85 % true pass predicted pass
  • 5 h, 65 % true fail predicted fail
  • test accuracy = 1.00
  • new student 6 h, 72 % -> fail
  • new student 11 h, 60 % -> fail

SUPERVISED WORKFLOW

  • 1 collect labelled data 2 split train / test
  • 3 train the model 4 test on unseen data
  • 5 predict new cases

k-NN: find the k closest known examples and take a majority vote - 'tell me who your friends are'.

Unsupervised Learning: Finding Groups

MODULE 2 · UNSUPERVISED

REAL OUTPUT: L24/UNSUPERVISED.PY

  • group 0: 6 students
  • centre 9.2 h, 85.5 %
  • group 1: 6 students
  • centre 3.8 h, 59.7 %
  • labels:
  • 1 1 1 1 1 1 0 0 0 0 0 0

K-MEANS (P06A)

Note: k-means finds the two groups but cannot name them. People decide what a group means.

  • repeat until no change:
  • assign each point to its nearest centre
  • move each centre to the mean of its points

Where You Have Already Used Them

MODULE 2 · LINKS TO THE PRACTICALS

Overfitting

Feature scaling

Memorising the training data. Always measure on a test set the model has not seen.

Attendance (50-95) swamps hours (2-12) in a distance. Scale first (StandardScaler, P04b).

Slide 12

MODULE 3

Reinforcement Learning

Learning by trial, error and reward

The Reinforcement Learning Loop

MODULE 3 · AGENT AND REWARDS

State s

Action a

Reward r

Policy

The situation now: the rider is at cell 2.

Move left or right.

-1 per move, +10 at the goal.

Best action in each state - what is learned.

THE RL LOOP

  • loop:
  • agent sees state s and chooses action a
  • environment returns reward r and new state s'
  • agent updates what it knows

Goal: the largest TOTAL reward over time, not only the next reward.

Q-Learning: The Update Rule

MODULE 3 · Q-LEARNING

Q-LEARNING UPDATE (WATKINS, 1989)

Q(s, a) <- Q(s, a) + α · [ r + γ · max Q(s', a') - Q(s, a) ]

REAL OUTPUT: L24/QUPDATE.PY - ONE UPDATE BY HAND

Q = 0.0 + 0.5 x (-1 + 0.9 x 4.0 - 0.0) = 1.30

Q-Learning on a Delivery Road

MODULE 3 · WORKED EXAMPLE

  • Early trips: wander - 10 and 13 steps.
  • Later trips: 4 steps, the shortest.
  • Q values: 8.00 = -1 + 0.9 x 10.00.
  • Policy: always move right.

REAL OUTPUT: L24/QLEARN.PY

  • steps per episode: 10 13 6 4 4 6 6 4 4 5 ... 4 5 4 4 7
  • learned Q table left right
  • cell 0 2.71 4.58
  • cell 1 0.95 6.20
  • cell 2 1.67 8.00
  • cell 3 3.45 10.00
  • policy: 0:right 1:right 2:right 3:right

Key idea: the reward at the goal flows back, one cell at a time, through γ.

Real uses: AlphaGo (2016) beat the world Go champion; robots learning to walk; traffic-light timing.

Activity: Which Kind of Learning?

CLASS ACTIVITY · PAIRS · 8 MIN

Pairs, 6 min + 2 min check. Students (hours, attendance %) - fail: (2, 55) (3, 60) (4, 50) (5, 65) (3, 70) (6, 58); pass: (8, 80) (9, 75) (7, 90) (10, 85) (12, 88) (9, 95)

Supervised, unsupervised or RL? (a) wheat yield from past records (b) group mobile users (c) a robot learns to walk (d) spam filter

Q-learning: Q = 2, r = -1, γ = 0.9, best next = 6.2, α = 0.5. New Q?

Classification or regression? (a) electricity bill in rupees (b) is a mango ripe?

What feedback does RL get that unsupervised learning does not?

k-NN, k = 3: new student 7 h, 72 %. Nearest three and the vote? (use the 12 students)

Who writes the rules in (a) the L23 fan controller (b) a k-NN model?

Common Mistakes

WATCH OUT

WRONG IDEA

CORRECT IDEA

✗ Unsupervised learning has a teacher.

✓ No labels: it finds groups by itself.

✗ Clustering tells us what the groups mean.

✓ People name the groups afterwards.

✗ 100 % training accuracy = a good model.

✓ Measure on unseen test data (overfitting).

✗ RL is told the correct action.

✓ It only gets rewards and learns by trial.

✗ Regression means 'going backwards'.

✓ Regression predicts a number.

Recap Quiz

CHECK YOUR UNDERSTANDING · 3 MIN

  • Q1. Learning from examples that include the correct answers is:
  • (a) Supervised (b) Unsupervised (c) Reinforcement (d) Fuzzy
  • Q2. Predicting tomorrow's temperature in °C is:
  • (a) Regression (b) Classification (c) Clustering (d) Reinforcement
  • Q3. k-means is an example of:
  • (a) Unsupervised learning (b) Supervised learning (c) Reinforcement learning (d) A fuzzy system
  • Q4. Q = 0, α = 0.5, r = 10, goal reached (no future). New Q:
  • (a) 5 (b) 10 (c) 0.5 (d) 9

Homework and Next Lecture

BEFORE THE NEXT CLASS

  • Task 1: k-NN with the 12 students: new student 8 h, 60 %. Find the nearest 1 and 3 students (distances) and the predictions. Is the answer fair? Why?
  • Task 2: Road of S15, all Q = 0, α = 0.5, γ = 0.9. (i) Rider at cell 2 moves right (r = -1). (ii) From cell 3 moves right to the goal (r = 10). (iii) Again from cell 2 right. Q after each?
  • Task 3: k-means, k = 2, on spending 2, 3, 4, 10, 11, 12 (thousand Rs) with starting centres 3 and 4. Show each iteration.

NEXT LECTURE

  • Lecture 25:
  • Decision Trees

← All CS-511 material