Nexus Learn

Lecture 20: Probabilistic Reasoning and Bayes' Rule

Lecture Slides CS-511 Artificial Intelligence 7 min read Updated 8 Nov 2026

↓ Download slides (PowerPoint)

Recap of Lecture 19 and Homework Check

LAST LECTURE · 5 MIN

Today: your dengue test is positive. The test is '90% accurate'. Are you 90% likely to have dengue? (Answer at S11 - most people are wrong.)

Today's Plan (50 minutes)

ROADMAP

0-5

Recap of Lecture 19 and homework check

S02

5-8

Plan and learning outcomes

S03-S04

8-17

Module 1: Probability basics - joint, marginal, conditional, independence

S05-S08

17-28

Module 2: Bayes' rule - the dengue test and the base rate

S09-S12

28-35

Module 3: Naive Bayes - an SMS spam filter

S13-S16

35-43

Class activity: conditional probability and Bayes

S17

43-48

Common mistakes and recap quiz

S18-S19

48-50

Homework and next lecture

S20

Learning Outcomes (SLOs)

BY THE END OF THIS LECTURE YOU WILL BE ABLE TO

SLO 20.1

Compute joint, marginal and conditional probabilities from a table and test independence

→ CLO3 · C3 Apply

SLO 20.2

Apply Bayes' rule to update a belief from evidence and explain the effect of the prior

→ CLO3 · C3 Apply

SLO 20.3

Apply a Naive Bayes classifier to a small example and explain its independence assumption

→ CLO3 · C3 Apply

  • Course Learning Outcome(s) served
  • CLO3: Apply various techniques and algorithms of AI used in general problem solving, optimization problems, constraint satisfaction problems, and game programming.

Slide 5

MODULE 1

Probability Basics

A number for 'how sure am I?'

Why Do AI Systems Need Probability?

MODULE 1 · UNCERTAINTY

Hidden causes

Noisy sensors

Too many exceptions

A doctor sees fever, not the virus. A farmer sees yellow stripes, not the fungus.

A cheap thermometer or a blurry camera gives wrong readings sometimes.

'Fever means dengue' - except flu, typhoid, malaria ... rules cannot list them all.

Joint, Marginal and Conditional Probability

MODULE 1 · FROM A TABLE

REAL OUTPUT: L20/JOINT.PY (MADE-UP CLINIC DATA)

  • fever no fever total
  • dengue 45 5 50
  • no dengue 190 760 950
  • total 235 765 1000
  • P(dengue) = 0.050 P(fever) = 0.235
  • P(dengue AND fever) = 0.045
  • P(dengue | fever) = 0.045 / 0.235 = 0.191
  • P(fever | dengue) = 0.045 / 0.050 = 0.900
  • P(dengue) x P(fever) = 0.0118, not 0.045
  • Independent? no

Not the same: P(fever | dengue) = 0.90 but P(dengue | fever) = 0.19. Most fever patients do NOT have dengue.

Rules of Probability and Independence

MODULE 1 · RULES

Independent

Dependent

Two coin tosses; rain in Burewala and a cricket result in Sydney.

Dengue and fever; clouds and rain; load-shedding and a hot day.

Slide 9

MODULE 2

Bayes' Rule

Turning 'symptom given disease' into 'disease given symptom'

Bayes' Rule

MODULE 2 · THE FORMULA

BAYES' RULE

  • P(E | H) x P(H)
  • P(H | E) = -----------------
  • P(E)
  • P(E) = P(E | H) P(H) + P(E | not H) P(not H)

In words: posterior = likelihood x prior / evidence. New belief = old belief, re-weighted by how well it explains what we saw.

Why: studies give P(symptom | disease); the patient wants P(disease | symptom). Bayes turns one into the other.

The Dengue Test: Positive - So How Worried?

MODULE 2 · WORKED EXAMPLE

18 test + (90%)

REAL OUTPUT: L20/BAYES.PY

  • Prior P(D) = 0.02
  • Sensitivity P(+|D) = 0.9
  • False positive P(+|not D) = 0.05
  • P(+) = 0.9 x 0.02 + 0.05 x 0.98
  • = 0.067
  • P(D|+) = 0.9 x 0.02 / 0.067 = 0.269
  • 20 dengue
  • (2%)

2 test -

  • 1,000
  • people

49 test + (5%)

Count: 67 positives, only 18 really ill: 18 / 67 = 0.27.

  • 980 healthy
  • (98%)

931 test -

Answer to S02: a '90% accurate' test that is positive means about 27% chance of dengue here - not 90%.

The Prior Matters: Base Rate and Repeated Tests

MODULE 2 · BASE RATE

REAL OUTPUT: L20/BAYES.PY

  • Second positive test: 0.269 -> 0.869
  • Effect of the prior (same test):
  • prior 0.001 -> P(D|+) = 0.018
  • prior 0.02 -> P(D|+) = 0.269
  • prior 0.2 -> P(D|+) = 0.818
  • prior 0.5 -> P(D|+) = 0.947

Base-rate fallacy: judging a test result without the prior. A rare disease stays fairly unlikely after one positive test.

Sequential: the posterior becomes the next prior: 0.02 -> 0.269 -> 0.869 after two positive tests.

Slide 13

MODULE 3

Naive Bayes

Many clues, one simple assumption

The Naive Bayes Classifier

MODULE 3 · IDEA

NAIVE BAYES WITH LAPLACE SMOOTHING

  • score(C) = P(C) x P(w1|C) x P(w2|C) x ... x P(wn|C)
  • P(w | C) = (count of w in class C + 1)
  • / (total words in C + vocabulary size)
  • choose the class with the larger score

Naive assumption

Clues are independent once the class is known.

Laplace (+1)

No word gets probability 0 just because it was not seen.

Used in: spam filters, news and complaint sorting, sentiment of reviews, quick medical screening - fast, needs little data.

Why it works: even when the probabilities are not exact, the class with the highest score is usually right.

Worked Example: An SMS Spam Filter

MODULE 3 · EXAMPLE

REAL OUTPUT: L20/NAIVE_BAYES.PY (MADE-UP PRACTICE MESSAGES)

  • Training: 4 spam, 6 ham; vocabulary 28 words; spam words 15, ham words 25
  • P('free'|spam) = (3+1)/(15+28) = 0.0930 P('free'|ham) = (1+1)/(25+28) = 0.0377
  • P('prize'|spam) = (2+1)/(15+28) = 0.0698 P('prize'|ham) = (1+1)/(25+28) = 0.0377
  • P('now'|spam) = (2+1)/(15+28) = 0.0698 P('now'|ham) = (1+1)/(25+28) = 0.0377
  • score spam = 1.811e-04 score ham = 3.224e-05
  • P(spam | 'free prize now') = 0.849
  • scikit-learn MultinomialNB: P(spam) = 0.849
  • 'see you after class': P(spam) = 0.088
  • 'win a free recharge': P(spam) = 0.937

Probability vs Certainty Factors

MODULE 3 · COMPARISON

Next lecture: Bayesian belief networks - a graph of causes and effects that makes probability practical for many variables.

Activity: Conditional Probability and Bayes

CLASS ACTIVITY · PAIRS · 8 MIN

Pairs, 6 min + 2 min check. Calculators allowed.

Clinic table (S07): P(no dengue | fever)?

Why does a 90% sensitive test give only 27% here?

Pakistan won the toss in 20 of 40 matches; won 14 of those 20 and 8 of the other 20. P(win)? P(win | toss won)? P(toss won | win)?

SMS 'see you after class': spam or ham? (S15)

Stripe rust in 10% of fields; stripes seen in 80% of rust fields and 10% of healthy. P(rust | stripes)?

P(A) = 0.3, P(B) = 0.5, P(A and B) = 0.15. Independent?

Common Mistakes

WATCH OUT

WRONG IDEA

CORRECT IDEA

✗ P(dengue | fever) = P(fever | dengue).

✓ 0.19 vs 0.90 - swapping the condition changes everything.

✗ A '90% accurate' positive test means 90% ill.

✓ Depends on the prior: 27% at 2%, 95% in an outbreak ward.

✗ P(A and B) = P(A) x P(B) always.

✓ Only if A and B are independent.

✗ Naive Bayes thinks words really are independent.

✓ It assumes it to simplify; ranking is still usually right.

✗ An unseen word can be given probability 0.

✓ Use Laplace (+1) smoothing - a zero wipes out the product.

Recap Quiz

CHECK YOUR UNDERSTANDING · 3 MIN

  • Q1. P(dengue | fever) means:
  • (a) Fever given dengue (b) Dengue given fever (c) Dengue and fever (d) Dengue or fever
  • Q2. Prior 0.02, sensitivity 0.9, false positive 0.05. P(D | +) is about:
  • (a) 0.90 (b) 0.27 (c) 0.05 (d) 0.67
  • Q3. A and B are independent if:
  • (a) P(A and B) = P(A) + P(B) (b) P(A | B) = P(A) (c) P(A) = P(B) (d) P(A | B) = 1
  • Q4. Laplace smoothing adds 1 to counts to avoid:
  • (a) Negative numbers (b) Zero probabilities (c) Large vocabularies (d) Overfitting priors

Homework and Next Lecture

BEFORE THE NEXT CLASS

  • Task 1: Over 30 days: rain and load-shedding 6, rain and none 2, no rain and load-shedding 9, no rain and none 13. Find P(LS), P(LS | rain), P(rain | LS). Independent?
  • Task 2: A mango-sorting camera: 15% of mangoes are bruised. It flags 95% of bruised and 8% of good mangoes. P(bruised | flagged)?
  • Task 3: With the S15 training messages, classify 'free notes' by Naive Bayes (Laplace +1). Spam or ham?

NEXT LECTURE

  • Lecture 21:
  • Bayesian Belief Networks

← All CS-511 material