Nexus Learn

Lecture 26: Neural Networks and the Perceptron

Lecture Slides CS-511 Artificial Intelligence 7 min read Updated 8 Nov 2026

↓ Download slides (PowerPoint)

Recap of Lecture 25 and Homework Check

LAST LECTURE · 5 MIN

Today: a tree asks one question at a time. A neuron weighs ALL its inputs at once and adds them up - like a judge giving marks.

Today's Plan (50 minutes)

ROADMAP

0-5

Recap of Lecture 25 and homework check

S02

5-8

Plan and learning outcomes

S03-S04

8-16

Module 1: From the brain to an artificial neuron

S05-S07

16-27

Module 2: The perceptron learning rule

S08-S11

27-35

Module 3: What one perceptron cannot do - XOR

S12-S14

35-43

Class activity: neurons and weight updates by hand

S15

43-48

Common mistakes and recap quiz

S16-S17

48-50

Homework and next lecture

S18

Learning Outcomes (SLOs)

BY THE END OF THIS LECTURE YOU WILL BE ABLE TO

SLO 26.1

Explain the parts of an artificial neuron and compute its output with a step activation

→ CLO3 · C2 Understand

SLO 26.2

Apply the perceptron learning rule to train a neuron on a small data set (AND, OR)

→ CLO3 · C3 Apply

SLO 26.3

Explain linear separability and why a single perceptron cannot learn XOR

→ CLO3 · C2 Understand

  • Course Learning Outcome(s) served
  • CLO3: Apply various techniques and algorithms of AI used in general problem solving, optimization problems, constraint satisfaction problems, and game programming.

Slide 5

MODULE 1

From the Brain to an Artificial Neuron

Inputs, weights, a sum and a decision

Biological and Artificial Neurons

MODULE 1 · THE MODEL

THE PERCEPTRON'S OUTPUT

Bias b: moves the cut-off - a negative bias makes the neuron harder to fire.

  • z = w1·x1 + w2·x2 + w3·x3 + b
  • y = step(z) = 1 if z >= 0 else 0

McCulloch and Pitts (1943) first model; Rosenblatt (1958) added learning - the perceptron.

One Neuron by Hand: Run the Tubewell Tonight?

MODULE 1 · WORKED EXAMPLE

REAL OUTPUT: L26/NEURON.PY

  • soil is dry (0/1) x = 1 w = +0.6 w*x = +0.6
  • hot day (0/1) x = 1 w = +0.4 w*x = +0.4
  • rain forecast (0/1) x = 0 w = -0.9 w*x = +0.0
  • bias b = -0.5
  • z = w.x + b = +0.5 -> step(z) = 1 (1 = run the pump)
  • same day with rain forecast: z = -0.4 -> 0

BY HAND

  • z = 0.6·1 + 0.4·1 + (-0.9)·0 + (-0.5) = +0.5 -> 1 (run)
  • z = 0.6·1 + 0.4·1 + (-0.9)·1 + (-0.5) = -0.4 -> 0 (wait)

Question: we chose these weights ourselves. Can the neuron learn them from past nights instead?

Slide 8

MODULE 2

The Perceptron Learning Rule

Learn from every mistake

The Learning Rule

MODULE 2 · ROSENBLATT (1958)

PERCEPTRON RULE

  • error = t - y
  • w <- w + lr · error · x
  • b <- b + lr · error

Learning rate

Epoch

Stop

Step size lr. We use lr = 1 so the numbers stay whole.

One pass over all the training examples.

When a whole epoch has no mistakes.

Convergence theorem: if one straight line can separate the classes, the rule is guaranteed to find one.

Training a Perceptron on AND

MODULE 2 · WORKED EXAMPLE

REAL OUTPUT: L26/TRAIN_AND.PY (LR = 1, START W = 0, B = 0)

  • x=(0, 0) t=0 y=1 -> w=(+0, +0) b=-1
  • x=(0, 1) t=0 y=0 -> w=(+0, +0) b=-1
  • x=(1, 0) t=0 y=0 -> w=(+0, +0) b=-1
  • x=(1, 1) t=1 y=0 -> w=(+1, +1) b=+0
  • epoch 1: 2 mistakes w = (1, 1), b = 0
  • epoch 2: 3 mistakes w = (2, 1), b = -1
  • epoch 3: 3 mistakes w = (2, 1), b = -2
  • epoch 4: 2 mistakes w = (2, 2), b = -2
  • epoch 5: 1 mistake w = (2, 1), b = -3
  • epoch 6: 0 mistakes w = (2, 1), b = -3
  • learned: w = (2, 1), b = -3 after 6 epochs

CHECK

  • check w = (2, 1), b = -3:
  • (1, 1): 2 + 1 - 3 = 0 -> 1
  • (1, 0): 2 + 0 - 3 = -1 -> 0
  • (0, 1): 0 + 1 - 3 = -2 -> 0

Epoch 1 by hand: (0, 0) gives z = 0 so y = 1, but t = 0: error -1, so b = -1 (w stays, because x = 0).

A Perceptron for the 12 Students

MODULE 2 · REAL DATA

CODE: L26/STUDENTS_PERC.PY

REAL OUTPUT

  • from sklearn.linear_model import Perceptron
  • from sklearn.preprocessing import StandardScaler
  • sc = StandardScaler().fit(X)
  • m = Perceptron(random_state=0).fit(sc.transform(X), y)
  • weights [1.32 0.69] bias 0.00
  • training accuracy 1.00 epochs 7
  • new student 8 h, 60 % -> pass
  • new student 4 h, 85 % -> fail
  • new student 11 h, 60 % -> pass
  • new student 7 h, 72 % -> pass

Boundary: always a straight line w1·x1 + w2·x2 + b = 0 - the tree's boundary was a step on one feature.

Lesson: both models are 100 % on 12 students but disagree on new ones. More data, and a test set, decide which to trust.

Slide 12

MODULE 3

What One Perceptron Cannot Do

Linear separability and XOR

AND and OR Yes - XOR No

MODULE 3 · LINEAR SEPARABILITY

REAL OUTPUT: L26/GATES.PY (LR 1, AT MOST 20 EPOCHS)

  • AND: target [0, 0, 0, 1] output [0, 0, 0, 1] w = (2, 1) b = -3 learned in 6 epochs
  • OR : target [0, 1, 1, 1] output [0, 1, 1, 1] w = (1, 1) b = -1 learned in 4 epochs
  • XOR: target [0, 1, 1, 0] output [1, 1, 0, 0] w = (-1, 0) b = 0 still 4 mistakes after 20 epochs

The Way Out: More Than One Layer

MODULE 3 · PREVIEW OF LECTURE 27

TWO LAYERS OF PERCEPTRONS

  • hidden layer:
  • h1 = OR(x1, x2)
  • h2 = NAND(x1, x2)
  • output layer:
  • y = AND(h1, h2) = XOR

Hidden layer

Smooth activation

Back-propagation

Neurons between input and output build new features.

Sigmoid instead of step, so errors have a slope.

Send the error backwards to every weight (L27).

You have used it: P02 built a perceptron; P03's MLPClassifier is exactly this - many layers of neurons.

Activity: Neurons and Weight Updates by Hand

CLASS ACTIVITY · PAIRS · 8 MIN

Pairs, 6 min + 2 min check. Step function: 1 if z >= 0, else 0. Tubewell neuron: w = (0.6, 0.4, -0.9), b = -0.5.

Tubewell neuron, inputs (dry, hot, rain): (1, 0, 0), (0, 1, 0), (1, 1, 1). Output of each?

Check that w = (-1, -1), b = 1.5 computes NAND.

w = (0, 0), b = 0, lr = 1, example x = (1, 1), t = 1. What changes?

Is the data 'pass if hours + attendance/10 >= 14' linearly separable? Why?

w = (1, 1), b = -1, lr = 1, example x = (0, 1), t = 0. New w and b?

Why can the perceptron rule not train the hidden neurons of the XOR network?

Common Mistakes

WATCH OUT

WRONG IDEA

CORRECT IDEA

✗ The perceptron changes weights after every example.

✓ Only when it makes a MISTAKE (t - y is not 0).

✗ If x = 0 its weight still changes.

✓ lr · error · 0 = 0: only weights of active inputs change.

✗ A perceptron can learn any pattern.

✓ Only linearly separable ones - not XOR.

✗ The bias is just another input value.

✓ It is a learned weight on a constant input 1.

✗ One correct epoch means the weights are unique.

✓ Many weight sets work; start and order change them.

Recap Quiz

CHECK YOUR UNDERSTANDING · 3 MIN

  • Q1. w = (2, 1), b = -3, x = (1, 1). The step output is:
  • (a) 0 (b) 1 (c) 3 (d) -3
  • Q2. In the perceptron rule, when y = t the weights:
  • (a) Increase (b) Decrease (c) Do not change (d) Reset to 0
  • Q3. Which gate can a single perceptron NOT learn?
  • (a) AND (b) OR (c) NAND (d) XOR
  • Q4. Who showed the XOR limit of perceptrons (1969)?
  • (a) Minsky and Papert (b) Rosenblatt (c) Quinlan (d) Turing

Homework and Next Lecture

BEFORE THE NEXT CLASS

  • Task 1: Train a perceptron for NOT (one input x, t = 1 for x = 0 and t = 0 for x = 1) with lr = 1, w = 0, b = 0. Show each update.
  • Task 2: Train OR by hand (lr = 1, start w = (0, 0), b = 0, order (0,0), (0,1), (1,0), (1,1)). Show the weights after each example until an epoch has no mistakes.
  • Task 3: Should a farmer sow cotton? Inputs (price high, rain expected, seed available) = (1, 0, 1), w = (0.5, 0.3, 0.4), b = -0.6. Compute z and the output.

NEXT LECTURE

  • Lecture 27:
  • Multilayer Networks and Back-propagation

← All CS-511 material