Learning Outcomes
SLO | After this lecture you will be able to... | CLO | Bloom |
|---|---|---|---|
SLO 24.1 | Explain machine learning and differentiate supervised, unsupervised and reinforcement learning with examples | CLO3 | C2 Understand |
SLO 24.2 | Apply k-nearest neighbours to labelled data and one k-means step to unlabelled data | CLO3 | C3 Apply |
SLO 24.3 | Apply the Q-learning update rule to a small reinforcement learning problem | CLO3 | C3 Apply |
1. What Is Machine Learning?
§1.1 Traditional programming vs machine learning
Slide L24-S06
Traditional: a person writes the rules (L19 expert system, L23 fan rules). Machine learning (ML): we give data and answers, and the computer finds the rules - a model - which then answers new cases. Arthur Samuel (1959): 'the ability to learn without being explicitly programmed'.
Traditional: rules + data -> answers Machine learning: data + answers -> rules (a model) model + new data -> answers |
Mitchell (1997): a program learns from experience E for task T, measured by P, if its performance on T (by P) improves with E. Spam filter: T = mark spam, E = emails users marked, P = % marked correctly.
§1.2 Three kinds of learning
Slide L24-S07
Supervised | Unsupervised | Reinforcement | |
|---|---|---|---|
Feedback | Correct answer (label) | None | Reward / penalty |
Output | Prediction | Groups, structure | Policy (what to do) |
Everyday picture | Solved past paper | Sorting mixed screws | Learning to ride a bicycle |
Example | Pass/fail, wheat yield | Customer groups | Robot walking, AlphaGo |
2. Supervised and Unsupervised Learning
Data used all lecture: 12 BSCS students (study hours per week, attendance %).
Result | Students (hours, attendance %) |
|---|---|
fail | (2, 55) (3, 60) (4, 50) (5, 65) (3, 70) (6, 58) |
pass | (8, 80) (9, 75) (7, 90) (10, 85) (12, 88) (9, 95) |
§2.1 Supervised learning
Slide L24-S09
Each example has features (inputs) and a label (answer). Classification: the label is a category (pass/fail, digit). Regression: the label is a number (electricity units). Workflow: collect labelled data, split into training and test sets, train, test on unseen data, predict new cases. k-nearest neighbours (k-NN): find the k closest known examples (distance √(Δx² + Δy²)) and take a majority vote.
trained on 8 students, tested on 4 9 h, 75 % true pass predicted pass 6 h, 58 % true fail predicted fail 10 h, 85 % true pass predicted pass 5 h, 65 % true fail predicted fail test accuracy = 1.00 new student 6 h, 72 % -> fail new student 11 h, 60 % -> fail |
Why is '11 h, 60 %' predicted fail? Attendance numbers (50-95) are much bigger than hours (2-12), so they dominate the distance. Feature scaling (StandardScaler, P04b) fixes this.
§2.2 Unsupervised learning
Slide L24-S10
group 0: 6 students centre 9.2 h, 85.5 % group 1: 6 students centre 3.8 h, 59.7 % labels: 1 1 1 1 1 1 0 0 0 0 0 0 |
k-means (P06a): choose k centres; repeat - assign each point to the nearest centre, move each centre to the mean of its points - until nothing changes. It finds groups but cannot name them; people give the groups a meaning. Uses: customer groups (Jazz, Zong), farms grouped by soil tests, unusual transactions (fraud).
§2.3 Links to the practicals
Slide L24-S11
Practical | What it learned | Kind |
|---|---|---|
P02, P03 | Perceptron / MLP classifier | Supervised (classification) |
P04a, P04b | Curve, electricity units | Supervised (regression) |
P05a, P05b | Handwritten digits | Supervised (classification) |
P06a | Household groups (k-means) | Unsupervised |
P06b | Tomorrow's electricity use | Supervised (regression) |
P11a, P11b | ANFIS Sugeno system | Supervised (regression) |
Overfitting: memorising the training data and failing on new data - always test on unseen data.
3. Reinforcement Learning
§3.1 The RL loop
Slide L24-S13
Word | Meaning | Delivery road |
|---|---|---|
Agent | The learner | The rider |
Environment | The world it acts in | The road |
State s | The situation now | Cell 2 |
Action a | What it does | Left or right |
Reward r | Number: + good, - bad | -1 per move, +10 at goal |
Policy | Best action in each state | Always right |
The aim is the largest total reward over time. Explore (try new actions) vs exploit (use the best known action).
§3.2 Q-learning
Slide L24-S14
Q(s, a) <- Q(s, a) + α · [ r + γ · max Q(s', a') - Q(s, a) ] |
α = learning rate (step size); γ = discount (how much the future counts); the bracket is the 'surprise'. At the goal there is no future, so max Q(s', a') = 0.
Q = 0.0 + 0.5 x (-1 + 0.9 x 4.0 - 0.0) = 1.30 |
§3.3 Worked example: the delivery road
Slide L24-S15
steps per episode: 10 13 6 4 4 6 6 4 4 5 ... 4 5 4 4 7 learned Q table left right cell 0 2.71 4.58 cell 1 0.95 6.20 cell 2 1.67 8.00 cell 3 3.45 10.00 policy: 0:right 1:right 2:right 3:right |
The reward at the goal flows back one cell at a time: 8.00 = -1 + 0.9 x 10.00, 6.20 = -1 + 0.9 x 8.00. Uses: AlphaGo (2016), robots learning to walk, traffic-light timing.
4. Common Mistakes
Wrong idea | Correct idea |
|---|---|
Unsupervised learning has a teacher. | No labels; it finds groups itself. |
Clustering names the groups. | People name them. |
100 % training accuracy = good model. | Measure on unseen test data. |
RL is told the correct action. | It only gets rewards. |
Regression means going backwards. | Regression predicts a number. |
Word Bank
Word | Simple meaning | Example |
|---|---|---|
Model | Rules learned from data | k-NN, MLP |
Feature | An input value | study hours |
Label | The correct answer of an example | pass |
Classification | Predicting a category | spam / not spam |
Regression | Predicting a number | units used |
Clustering | Grouping without labels | k-means |
Overfitting | Memorising instead of learning | 100 % train, 60 % test |
Reward | Score for an action | +10 at goal |
Policy | What to do in each state | always right |
Self-Check Questions
- Give Mitchell's T, E, P for a wheat-yield predictor. (see §1.1)
- Name the kind of learning: grouping cotton farms by soil tests. (see §1.2)
- Classification or regression: predicting a petrol price? (see §2.1)
- k-NN, k = 1: predict 4 h, 72 %. (see §2.1)
- Why can k-means not tell us which group is 'pass'? (see §2.2)
- Q = 1, r = -1, γ = 0.9, best next 8, α = 0.5: new Q? (see §3.2)
- Why is Q(cell 1, right) = 6.20? (see §3.3)
Reading: Kaplan Ch 2; Russell & Norvig Ch 19 (learning from examples) and Ch 22 (reinforcement learning); scikit-learn user guide.