Nexus Learn

Lecture 30: Natural Language Processing Basics

Lecture Notes CS-511 Artificial Intelligence 7 min read Updated 8 Nov 2026

↓ Download notes (Word)

On this page
  1. Learning Outcomes
  2. 1. What Is NLP?
  3. 2. Turning Text into Numbers
  4. 3. Sentiment Analysis and Language Models
  5. 4. Common Mistakes
  6. Word Bank
  7. Self-Check Questions

Learning Outcomes

SLO

After this lecture you will be able to...

CLO

Bloom

SLO 30.1

Explain what natural language processing is, name its main tasks and applications, and explain why language is hard for computers

CLO1

C2 Understand

SLO 30.2

Turn short texts into numbers by hand: tokens, stop words, a bag of words and TF-IDF

CLO1

C3 Apply

SLO 30.3

Explain how naive Bayes classifies the sentiment of a review, and how modern language models such as ChatGPT go beyond word counts

CLO1

C2 Understand

1. What Is NLP?

§1.1 Language tasks all around us

Slide L30-S06

Natural language processing (NLP) is the part of AI that makes computers read, understand and produce human language. A natural language is one that people speak - Urdu, English, Punjabi - not a programming language.

Task

Input

Output

Example

Text classification / sentiment

A message or review

Spam / not spam; positive / negative

Gmail, Daraz reviews

Machine translation

Urdu sentence

English sentence

Google Translate

Speech recognition

Voice

Text

Urdu voice typing

Information extraction

A document

Names, dates, amounts

Reading a bill

Question answering / chatbots

A question

An answer

ChatGPT, Gemini

§1.2 Why language is hard for computers

Slide L30-S07

The main difficulty is ambiguity (one sentence, more than one meaning). People use context and world knowledge; a computer must learn them from data.

Problem

Example

Why it is hard

Word with two meanings

bank (money / river); aam (mango / common)

Needs context

Sentence structure

I saw a man with a telescope

Who has the telescope?

Negation

The biryani was not bad

'bad', but the meaning is good

Sarcasm

Wah, kya service hai - 2 ghante intezar!

Positive words, angry meaning

Spelling variety

acha / achha / achaa

Same word, different tokens

Code-mixing

Service was bohat slow

Two languages in one sentence

2. Turning Text into Numbers

§2.1 Tokens and stop words

Slide L30-S09

Cleaning: lower case and remove punctuation. Tokenisation: split the text into tokens (usually words). Stop words are very common words that carry little meaning (the, is, and; Urdu: ka, ki, hai, aur) and are often removed.

Sentence : The new mobile is very good, but the battery is NOT good!

Tokens : ['the', 'new', 'mobile', 'is', 'very', 'good', 'but', 'the', 'battery', 'is', 'not', 'good'] (12)

Our stop words removed: ['new', 'mobile', 'good', 'battery', 'not', 'good']

scikit-learn list removed: ['new', 'mobile', 'good', 'battery', 'good'] ('not' is in its list!)

Roman Urdu, same meaning: [['khana', 'bohat', 'acha', 'tha'], ['khana', 'bht', 'achha', 'tha']]

Warning: scikit-learn's ready-made English stop-word list contains 'not', so 'battery is NOT good' becomes 'battery good'. For sentiment analysis, keep negation words. Roman Urdu has no fixed spelling, so real systems first normalise spellings (bht -> bohat).

§2.2 Bag of words

Slide L30-S10

Make a vocabulary of every different word, then describe each text by how many times each word appears. Word order is lost: 'dog bites man' and 'man bites dog' give the same bag. df (document frequency) = number of texts that contain the word.

Reviews (stop words 'was', 'and' removed):

R1: food good service good

R2: food cold service slow

R3: good biryani good price

Bag of words (counts)

word R1 R2 R3 df idf

biryani 0 0 1 1 0.477

cold 0 1 0 1 0.477

food 1 1 0 2 0.176

good 2 0 2 2 0.176

price 0 0 1 1 0.477

service 1 1 0 2 0.176

slow 0 1 0 1 0.477

§2.3 TF-IDF

Slide L30-S11

TF = count of the word in this text

IDF = log10( N / df ) N = number of texts, df = texts containing the word

TF-IDF = TF x IDF

TF-IDF of R1 (TF x IDF):

food 1 x 0.176 = 0.176

good 2 x 0.176 = 0.352

service 1 x 0.176 = 0.176

TF-IDF of R3: biryani 0.477, good 0.352, price 0.477

A word that appears in every text gets IDF = log10(1) = 0 - it cannot tell texts apart. A rare word gets a high IDF. Example: 'good' is in 2 of 3 reviews, IDF = log10(3/2) = 0.176, and appears twice in R1, so its TF-IDF there is 0.352; 'biryani' is in one review only, IDF = 0.477.

3. Sentiment Analysis and Language Models

§3.1 Sentiment analysis with naive Bayes

Slide L30-S13

Sentiment analysis decides whether a text is positive or negative. We use the naive Bayes method of Lecture 20: P(word | class) = (count + 1) / (words in the class + vocabulary size); multiply the word probabilities for each class and compare. Made-up practice reviews, 8 positive and 8 negative, in English and Roman Urdu:

Training: 8 positive, 8 negative reviews; vocabulary 41; positive words 36, negative words 34

P('khana'|pos) = (1+1)/(36+41) = 0.0260 P('khana'|neg) = (1+1)/(34+41) = 0.0267

P('acha'|pos) = (2+1)/(36+41) = 0.0390 P('acha'|neg) = (0+1)/(34+41) = 0.0133

P('tha'|pos) = (1+1)/(36+41) = 0.0260 P('tha'|neg) = (1+1)/(34+41) = 0.0267

P(positive | 'khana acha tha') = 0.735

scikit-learn, single words (41 features):

khana acha tha P(positive) = 0.735 -> positive

service was slow and food cold P(positive) = 0.243 -> negative

food was not good P(positive) = 0.458 -> negative

biryani was not bad P(positive) = 0.070 -> negative

scikit-learn, words + word pairs (92 features):

khana acha tha P(positive) = 0.844 -> positive

service was slow and food cold P(positive) = 0.238 -> negative

food was not good P(positive) = 0.211 -> negative

biryani was not bad P(positive) = 0.024 -> negative

Adding word pairs (bigrams such as 'not good') makes 'food was not good' clearly negative (0.458 -> 0.211). But 'biryani was not bad' - a good review - is still called negative, because 'not bad' never appeared in training. Word counting has limits.

§3.2 From word counts to ChatGPT

Slide L30-S14

Era

Idea

Example

1966 Rules

Hand-written patterns

ELIZA chatbot (MIT)

1990s Statistics

Word counts, naive Bayes, n-grams

Spam filters

2013 Word embeddings

Each word = a list of numbers; similar words are close

word2vec: king - man + woman = queen

2014-16 RNN / LSTM

Read words in order (Lecture 28)

Google Translate 2016

2017- Transformers

Attention over all words at once

ChatGPT, Gemini, Claude

A language model predicts the next word ('Pakistan ne match ...' -> 'jeet liya'). Trained on huge amounts of text, it learns grammar, facts and expressions such as 'not bad'. Urdu is a low-resource language (less text online), so Urdu NLP is an open research area.

4. Common Mistakes

Wrong idea

Correct idea

Computers understand words like we do.

They work with numbers made from words.

Always remove every stop word.

Keep negation words ('not', 'nahi') for sentiment.

Bag of words keeps word order.

Order is lost.

A high count always means an important word.

Words in every text get IDF 0.

ChatGPT looks words up in a dictionary.

It is a language model that predicts the next word.

Word Bank

Word

Simple meaning

Example

NLP

AI for human language

Google Translate

Token

One piece of text, usually a word

khana | acha | tha

Stop word

Very common word with little meaning

the, is, ka, hai

Vocabulary

All different words in a collection

7 words

Bag of words

Word counts, order ignored

R1 = 0 0 1 2 0 1 0

TF-IDF

Count weighted by how rare the word is

biryani 0.477

Sentiment

Feeling of a text: positive or negative

angry review

Ambiguity

More than one possible meaning

bank

Language model

Predicts the next word

ChatGPT

Self-Check Questions

  1. Name four NLP tasks with a Pakistani example of each. (see §1.1)
  2. Give three kinds of ambiguity with examples. (see §1.2)
  3. Why should 'not' be kept for sentiment analysis? (see §2.1)
  4. Make a bag of words for 'pani pani pani' and 'pani nahi'. (see §2.2)
  5. 4 texts; a word is in 2 of them. Find its IDF. (see §2.3)
  6. Why did word pairs help with 'food was not good'? (see §3.1)
  7. What does a language model predict? (see §3.2)

Reading: Russell & Norvig Ch 23 (natural language processing) and Ch 24 (deep learning for NLP); Lecture 20 (naive Bayes); Lecture 28 (RNNs).

← All CS-511 material