Recap of Lecture 20 and Homework Check
LAST LECTURE · 5 MIN
Today: 20 yes/no variables need a table of over a million numbers. A Bayesian network needs about 80. How?
Today's Plan (50 minutes)
ROADMAP
0-5
Recap of Lecture 20 and homework check
S02
5-8
Plan and learning outcomes
S03-S04
8-18
Module 1: Structure - nodes, arrows, CPTs and the chain rule
S05-S08
18-30
Module 2: Inference - enumeration, explaining away, sampling
S09-S12
30-35
Module 3: Building and using networks
S13-S14
35-43
Class activity: the weather network
S15-S16
43-48
Common mistakes and recap quiz
S17-S18
48-50
Homework and next lecture
S19
Learning Outcomes (SLOs)
BY THE END OF THIS LECTURE YOU WILL BE ABLE TO
SLO 21.1
Explain the structure of a Bayesian network (nodes, arrows, CPTs) and why it needs far fewer numbers than a full joint table
→ CLO3 · C2 Understand
SLO 21.2
Apply the chain rule to compute a joint probability from a network
→ CLO3 · C3 Apply
SLO 21.3
Apply inference by enumeration to a small network and explain explaining away
→ CLO3 · C3 Apply
- Course Learning Outcome(s) served
- CLO3: Apply various techniques and algorithms of AI used in general problem solving, optimization problems, constraint satisfaction problems, and game programming.
Slide 5
MODULE 1
Network Structure
Arrows from causes to effects
Why Bayesian Networks?
MODULE 1 · THE IDEA
Nodes and arrows
A CPT per node
Few numbers
One node per variable; an arrow from each direct cause to its effect. No loops (a DAG).
P(node | its parents) for every combination of parent values.
Each node only looks at its parents - not at every other variable.
REAL OUTPUT: L21/JOINT.PY - NUMBERS NEEDED
- 10 yes/no variables, max 2 parents each: network <= 40, full joint 1,023
- 20 yes/no variables, max 2 parents each: network <= 80, full joint 1,048,575
- 30 yes/no variables, max 2 parents each: network <= 120, full joint 1,073,741,823
DAG = directed acyclic graph: arrows have a direction and you can never follow them back to where you started.
The Shop Alarm Network
MODULE 1 · EXAMPLE
Theft (B)
Earthquake (E)
Alarm (A)
Junaid calls (J)
Maryam calls (M)
Numbers: Russell & Norvig, Fig. 13.2
10 numbers: 1 + 1 + 4 + 2 + 2 - instead of 31 for the full joint table.
The Chain Rule for Networks
MODULE 1 · JOINT PROBABILITY
CHAIN RULE FOR A BAYESIAN NETWORK
- P(x1, x2, ..., xn) = P(x1 | parents(x1)) x ... x P(xn | parents(xn))
- P(J, M, A, B, E) = P(J | A) P(M | A) P(A | B, E) P(B) P(E)
REAL OUTPUT: L21/JOINT.PY - A FALSE ALARM REPORTED BY BOTH
- P(j, m, a, not b, not e)
- = P(j|a) P(m|a) P(a|not b, not e) P(not b) P(not e)
- = 0.90 x 0.70 x 0.001 x 0.999 x 0.998
- = 0.000628
Tip: write the product first, then fill in the CPT numbers; for 'not' values use 1 - p (P(not b) = 0.999).
Slide 9
MODULE 2
Inference
Asking questions of the network
Inference by Enumeration
MODULE 2 · EXACT INFERENCE
ENUMERATION (BAYESNET.PY)
REAL OUTPUT: L21/INFER.PY
- P(B | j, m) = alpha x SUM over e, a of
- P(B) P(e) P(a | B, e) P(j | a) P(m | a)
- alpha = 1 / (value for B=true + value for B=false)
- P(theft | Junaid and Maryam call) = 0.284
- P(theft | only Junaid calls) = 0.016
- P(earthquake | both call) = 0.176
- P(theft | alarm) = 0.374
- P(theft | alarm, earthquake) = 0.003
- Unnormalised: true 0.0005922, false 0.0014919
- alpha = 1 / 0.0020841
Each clue raises the belief: 0.001 -> 0.016 -> 0.284. Still only 28% - thefts are rare.
Three Connections and Explaining Away
MODULE 2 · HOW EVIDENCE FLOWS
REAL OUTPUT: L21/INFER.PY
Explaining away: the earthquake explains the alarm, so belief in theft falls from 0.374 to 0.003.
- P(theft | alarm) = 0.374
- P(theft | alarm, earthquake) = 0.003
Daily life: your phone did not charge overnight - learning there was load-shedding 'explains away' a broken charger.
Approximate Inference: Sampling
MODULE 2 · BIG NETWORKS
REAL OUTPUT: L21/SAMPLING.PY (SEED 1)
- 10,000 samples: kept 24 estimate 0.250
- 100,000 samples: kept 213 estimate 0.286
- 1,000,000 samples: kept 2,109 estimate 0.275
- exact (enumeration): 0.284
Simulate
Draw B, E, then A, then J and M from their CPTs.
Reject
Throw away samples that do not match the evidence.
Trade-off: exact but slow (enumeration) vs fast but approximate (sampling). Rare evidence wastes most samples: only 2,109 of 1,000,000 kept.
Count
Fraction of kept samples with a theft.
Slide 13
MODULE 3
Building and Using Networks
From experts, from data, in real systems
Building a Network and Where They Are Used
MODULE 3 · PRACTICE
Medicine
Troubleshooting
Naive Bayes
Diagnosis networks with hundreds of diseases and symptoms.
Printer and computer fault finders ask the most useful question next.
The simplest network: one class node pointing at every word (L20).
Activity Network: Clouds, Rain and a Wet Road
CLASS ACTIVITY · USE THIS NETWORK
Cloudy (C)
Rain (R)
Wet road (W)
Activity: The Weather Network
CLASS ACTIVITY · PAIRS · 8 MIN
Pairs, 6 min + 2 min check. Use the network on the previous slide; item 6 uses S11.
P(c, r, w) - cloudy, rain and a wet road - by the chain rule.
The road is wet. P(r | w)? (Bayes)
P(r): add over cloudy and not cloudy.
How many numbers does this network need? The full joint?
P(w): add over rain and no rain.
Alarm network: is a theft more or less likely once we hear of an earthquake? Why?
Common Mistakes
WATCH OUT
WRONG IDEA
CORRECT IDEA
✗ Arrows can point any way, loops are fine.
✓ A DAG: arrows from causes to effects, no loops.
✗ Each node's CPT lists all other variables.
✓ Only its parents - that is why networks are small.
✗ P(theft | both call) is high because both called.
✓ Only 0.284: thefts are rare (prior 0.001).
✗ Forget to normalise after enumeration.
✓ Divide by the sum for true and false (alpha).
✗ Learning one cause never changes another.
✓ In a collider it does: explaining away (0.374 -> 0.003).
Recap Quiz
CHECK YOUR UNDERSTANDING · 3 MIN
- Q1. A Bayesian network is a:
- (a) Directed acyclic graph with CPTs (b) Neural network (c) Decision tree (d) Search tree
- Q2. In the alarm network, the CPT of Alarm has how many rows?
- (a) 1 (b) 2 (c) 4 (d) 8
- Q3. P(J, M, A, B, E) factorises as P(J|A) P(M|A) P(A|B,E) times:
- (a) P(B) P(E) (b) P(B|E) (c) P(A) (d) P(J) P(M)
- Q4. Theft -> Alarm <- Earthquake is a:
- (a) Chain (b) Fork (c) Collider (d) Loop
Homework and Next Lecture
BEFORE THE NEXT CLASS
- Task 1: Alarm network: compute P(not j, m, a, b, not e) by the chain rule.
- Task 2: Weather network: the road is wet. P(cloudy | wet road)?
- Task 3: Draw a dengue network: Monsoon -> Mosquitoes -> Dengue -> Fever and LowPlatelets. Choose CPTs: P(monsoon) 0.3; mosquitoes 0.8 / 0.2; dengue 0.05 / 0.005; fever 0.95 / 0.1; low platelets 0.8 / 0.05. Count the numbers; find P(dengue).
NEXT LECTURE
- Lecture 22:
- Fuzzy Sets and Fuzzy Logic