Recap of Lecture 29 and Homework Check
LAST LECTURE · 5 MIN
Today: a new kind of data - words. How does Gmail know a message is spam, or a shop know its reviews are angry?
Today's Plan (50 minutes)
ROADMAP
0-5
Recap of Lecture 29 and homework check
S02
5-8
Plan and learning outcomes
S03-S04
8-15
Module 1: What NLP is and why language is hard
S05-S07
15-27
Module 2: Turning text into numbers
S08-S11
27-35
Module 3: Sentiment analysis and language models
S12-S14
35-43
Class activity: text to numbers by hand
S15
43-48
Common mistakes and recap quiz
S16-S17
48-50
Homework and next lecture
S18
Learning Outcomes (SLOs)
BY THE END OF THIS LECTURE YOU WILL BE ABLE TO
SLO 30.1
Explain what natural language processing is, name its main tasks and applications, and explain why language is hard for computers
→ CLO1 · C2 Understand
SLO 30.2
Turn short texts into numbers by hand: tokens, stop words, a bag of words and TF-IDF
→ CLO1 · C3 Apply
SLO 30.3
Explain how naive Bayes classifies the sentiment of a review, and how modern language models such as ChatGPT go beyond word counts
→ CLO1 · C2 Understand
- Course Learning Outcome(s) served
- CLO1: Explain the applications of artificial intelligence and the agent-based approach to AI.
Slide 5
MODULE 1
What Is NLP?
Computers and human language
Language Tasks All Around Us
MODULE 1 · APPLICATIONS
Classification
Translation
Speech to text
Chatbots
Spam filter; angry or happy review.
Urdu to English and back.
Urdu voice typing; voice notes.
Bank helpline; ChatGPT, Gemini.
Natural language = a language people speak (Urdu, English, Punjabi) - not a programming language.
Why Language Is Hard for Computers
MODULE 1 · AMBIGUITY
Ambiguity: one sentence, more than one meaning. People use context and world knowledge; a computer has to learn them from data.
Slide 8
MODULE 2
Turning Text into Numbers
Models need numbers, not words
Step 1: Tokens and Stop Words
MODULE 2 · PRE-PROCESSING
REAL OUTPUT: L30/TOKENS.PY
- Sentence : The new mobile is very good, but the battery is NOT good!
- Tokens : ['the', 'new', 'mobile', 'is', 'very', 'good', 'but', 'the', 'battery', 'is', 'not', 'good'] (12)
- Our stop words removed: ['new', 'mobile', 'good', 'battery', 'not', 'good']
- scikit-learn list removed: ['new', 'mobile', 'good', 'battery', 'good'] ('not' is in its list!)
- Roman Urdu, same meaning: [['khana', 'bohat', 'acha', 'tha'], ['khana', 'bht', 'achha', 'tha']]
Careful: a ready-made stop-word list removed 'not' - and 'NOT good' became 'good'. Keep negation words for sentiment.
Step 2: Bag of Words
MODULE 2 · COUNTING WORDS
REAL OUTPUT: L30/BOW.PY
- Reviews (stop words 'was', 'and' removed):
- R1: food good service good
- R2: food cold service slow
- R3: good biryani good price
- Bag of words (counts)
- word R1 R2 R3 df idf
- biryani 0 0 1 1 0.477
- cold 0 1 0 1 0.477
- food 1 1 0 2 0.176
- good 2 0 2 2 0.176
- price 0 0 1 1 0.477
- service 1 1 0 2 0.176
- slow 0 1 0 1 0.477
Vocabulary
Every different word in all the reviews: 7 here.
One row per text
R1 = 0 0 1 2 0 1 0 - a vector of counts.
Order is lost
'dog bites man' = 'man bites dog'.
Picture: tip the words of a review into a bag and count them.
Step 3: TF-IDF - Which Words Matter?
MODULE 2 · WEIGHTING WORDS
FORMULA (N TEXTS, DF = TEXTS WITH THE WORD)
BOARD
- TF = count of the word in this text
- IDF = log10( N / df )
- TF-IDF = TF x IDF
- 'good': df = 2, IDF = log10(3/2) = 0.176
- R1: 2 x 0.176 = 0.352
- 'biryani': df = 1, IDF = log10(3/1) = 0.477
REAL OUTPUT: L30/BOW.PY
- TF-IDF of R1 (TF x IDF):
- food 1 x 0.176 = 0.176
- good 2 x 0.176 = 0.352
- service 1 x 0.176 = 0.176
- TF-IDF of R3: biryani 0.477, good 0.352, price 0.477
Idea: a word that appears everywhere tells us nothing; a rare word tells us a lot about the text.
Slide 12
MODULE 3
Sentiment and Language Models
Is this review happy or angry?
Sentiment Analysis with Naive Bayes
MODULE 3 · TEXT CLASSIFICATION
REAL OUTPUT: L30/SENTIMENT.PY (MADE-UP PRACTICE REVIEWS)
- Training: 8 positive, 8 negative reviews; vocabulary 41; positive words 36, negative words 34
- P('khana'|pos) = (1+1)/(36+41) = 0.0260 P('khana'|neg) = (1+1)/(34+41) = 0.0267
- P('acha'|pos) = (2+1)/(36+41) = 0.0390 P('acha'|neg) = (0+1)/(34+41) = 0.0133
- P('tha'|pos) = (1+1)/(36+41) = 0.0260 P('tha'|neg) = (1+1)/(34+41) = 0.0267
- P(positive | 'khana acha tha') = 0.735
- scikit-learn, single words (41 features):
- khana acha tha P(positive) = 0.735 -> positive
- service was slow and food cold P(positive) = 0.243 -> negative
- food was not good P(positive) = 0.458 -> negative
- biryani was not bad P(positive) = 0.070 -> negative
- scikit-learn, words + word pairs (92 features):
- khana acha tha P(positive) = 0.844 -> positive
- service was slow and food cold P(positive) = 0.238 -> negative
- food was not good P(positive) = 0.211 -> negative
- biryani was not bad P(positive) = 0.024 -> negative
Word pairs help: 'food was not good' 0.458 -> 0.211 once 'not good' is a feature.
Still wrong: 'biryani was not bad' is a good review, but both models say negative - 'not bad' was never seen.
From Word Counts to ChatGPT
MODULE 3 · LANGUAGE MODELS
Language model: predicts the next word. 'Pakistan ne match ...' -> 'jeet liya'. Trained on huge text, it learns grammar, facts and 'not bad'.
Urdu is a low-resource language (less text online) - Urdu NLP is an open research area.
Activity: Text to Numbers by Hand
CLASS ACTIVITY · PAIRS · 8 MIN
Pairs, 6 min + 2 min check. Lower case, remove punctuation, split on spaces.
Tokenise 'Match was GREAT, Pakistan won!' How many tokens?
3 texts. IDF of a word in 1 text? In all 3 texts?
Remove the stop word 'was'. How many tokens are left?
Name the task: Urdu voice typing; Gmail spam folder; Google Translate.
Bag of words for S1 'pakistan won match' and S2 'india won match match'.
Why is it dangerous to remove 'not' before sentiment analysis?
Common Mistakes
WATCH OUT
WRONG IDEA
CORRECT IDEA
✗ Computers understand words like we do.
✓ They work with numbers made from words (counts, TF-IDF, embeddings).
✗ Always remove every stop word.
✓ Keep negation words ('not', 'nahi') for sentiment.
✗ Bag of words keeps the word order.
✓ Order is lost: 'dog bites man' = 'man bites dog'.
✗ A high count always means an important word.
✓ Words in every text get IDF 0 - TF-IDF fixes this.
✗ ChatGPT looks words up in a dictionary.
✓ It is a language model that predicts the next word.
Recap Quiz
CHECK YOUR UNDERSTANDING · 3 MIN
- Q1. Splitting text into words is called:
- (a) Stemming (b) Tokenisation (c) Translation (d) Crossover
- Q2. 5 texts; a word appears in all 5. Its IDF is:
- (a) 1 (b) 5 (c) 0.699 (d) 0
- Q3. 'food good good' with vocabulary (bad, food, good) as a bag of words:
- (a) 0 1 1 (b) 1 1 2 (c) 0 1 2 (d) 2 1 0
- Q4. Deciding whether a review is happy or angry is:
- (a) Sentiment analysis (b) Machine translation (c) Speech recognition (d) Question answering
Homework and Next Lecture
BEFORE THE NEXT CLASS
- Task 1: Three SMS: 'free prize win', 'win the match', 'free tickets free'. Make the bag-of-words table with df and IDF = log10(3 / df) for every word.
- Task 2: Find the TF-IDF of 'free' in SMS 3, 'win' in SMS 1 and 'tickets' in SMS 3. Which word best describes SMS 3?
- Task 3: Review 'bura khana'. Positive words 36, negative words 34, vocabulary 41. 'bura' 0 times positive, 1 negative; 'khana' 1 and 1. Find P(positive) with +1 smoothing and equal priors.
NEXT LECTURE
- Lecture 31:
- Machine Vision