Learning Outcomes
SLO | After this lecture you will be able to... | CLO | Bloom |
|---|---|---|---|
SLO 4.1 | Define agent, sensor, actuator, percept sequence, agent function and agent program | CLO1 | C1 Remember |
SLO 4.2 | Explain rationality, performance measures, autonomy and why rational ≠ perfect | CLO1 | C2 Understand |
SLO 4.3 | Specify the PEAS description of a task environment | CLO1 | C2 Understand |
SLO 4.4 | Classify task environments by their seven properties | CLO1 | C2 Understand |
SLO 4.5 | Compare the five agent types and choose a suitable type for a task | CLO1 | C2 Understand |
1. Agents
§1.1 Agent and environment
Slide L04-S06 to S07
Definition: An agent is anything that perceives its environment through sensors and acts upon that environment through actuators. |
Agent | Sensors (input) | Actuators (output) |
|---|---|---|
Human | Eyes, ears, nose, skin | Hands, legs, voice |
Robot vacuum | Camera, bump sensor, dirt sensor | Wheels, brush, suction |
Self-driving car | Cameras, radar, LiDAR, GPS, speedometer | Steering, accelerator, brake, horn |
Careem dispatch app (software) | Drivers' GPS, ride requests, traffic data | Assign driver, show fare, send messages |
Spam filter (software) | E-mail text and sender | Move to inbox or spam folder |
Smart irrigation | Soil-moisture sensor, weather forecast | Open or close the tubewell valve |
A percept is what the agent senses at one moment (one camera frame, one e-mail).
§1.2 Percept sequence, agent function and agent program
Slide L04-S08
Term | Meaning |
|---|---|
Percept sequence | The complete history of everything the agent has perceived so far |
Agent function | A mathematical mapping from every possible percept sequence to an action (a table on paper, possibly infinite) |
Agent program | The real code that implements the agent function on the agent's hardware |
Architecture | The hardware (computer, sensors, actuators) the program runs on. Agent = architecture + program |
Vacuum world: two squares A and B, each Clean or Dirty. A percept is [location, status]. Actions: Left, Right, Suck, NoOp (do nothing). Part of the agent function:
Percept sequence | Action |
|---|---|
[A, Clean] | Right |
[A, Dirty] | Suck |
[B, Clean] | Left |
[B, Dirty] | Suck |
[A, Clean], [A, Clean] | Right |
[A, Clean], [A, Dirty] | Suck |
2. Rationality and PEAS
§2.1 Rational agents
Slide L04-S10
Definition: For each possible percept sequence, a rational agent selects an action that is expected to maximise its performance measure, given the evidence of the percept sequence and its built-in knowledge. |
What is rational at any time depends on four things: (1) the performance measure, (2) the agent's prior knowledge of the environment, (3) the actions it can perform, and (4) its percept sequence so far.
- Rational ≠ omniscient: an omniscient agent knows the actual future outcome. A rational agent cannot. If you cross a road carefully and a sign board falls on you, you still acted rationally.
- Information gathering: looking both ways before crossing is a rational action because it improves future decisions.
- Learning: a rational agent should learn from what it perceives.
- Autonomy: it should rely on its own percepts and learning, not only on rules built in by the designer.
§2.2 Designing performance measures
Slide L04-S11
Rule: measure what you actually want in the environment, not how you think the agent should behave.
Agent | Bad measure (can be cheated) | Better measure |
|---|---|---|
Robot vacuum | Amount of dirt sucked (it could dump dirt and suck it again) | Clean squares at every time step, minus energy used |
Careem dispatch | Number of rides assigned | Waiting time, fair fares, safety and rider ratings |
School | Pass percentage only | What students actually learned |
§2.3 PEAS
Slide L04-S12 to S13
PEAS stands for Performance measure, Environment, Actuators, Sensors. Writing PEAS is always the first step in designing an agent.
Agent | Performance | Environment | Actuators | Sensors |
|---|---|---|---|---|
Automated taxi in Lahore | Safe, fast, legal, comfortable trip; profit; low fuel | Roads, cars, rickshaws, motorbikes, pedestrians, potholes, fog, customers | Steering, accelerator, brake, indicator, horn, screen/speaker | Cameras, radar, LiDAR, GPS, speedometer, engine sensors, microphone |
Dengue diagnosis system | Correct diagnosis, healthy patient, low cost | Patient, doctor, hospital, lab | Screen showing questions, tests, diagnosis | Typed symptoms, lab results |
Crop-spraying drone | Pests killed, little poison wasted, crop safe | Cotton field, wind, plants, birds | Propellers, spray nozzle | Camera, GPS, wind sensor, tank level |
Daraz recommender | Sales, clicks, satisfaction | Website, customers, product list | Show suggestions | Clicks, searches, purchases, ratings |
3. Properties of Task Environments
§3.1 The seven properties
Slide L04-S15
Property | Question to ask | Example |
|---|---|---|
Fully vs partially observable | Do the sensors show everything relevant to the choice? | Chess board (fully); driving in fog, card games (partially) |
Single vs multi-agent | Are other agents acting? (competitive or cooperative) | Crossword (single); chess, cricket, traffic (multi) |
Deterministic vs stochastic | Is the next state completely decided by the current state and action? | Crossword (deterministic); Ludo dice, weather (stochastic) |
Episodic vs sequential | Do the current decisions affect future decisions? | Checking separate X-rays (episodic); chess, driving (sequential) |
Static vs dynamic | Does the world change while the agent is thinking? (semi-dynamic: only the score changes) | Crossword (static); chess with clock (semi); traffic (dynamic) |
Discrete vs continuous | Are states, time and actions countable or smoothly varying? | Chess moves (discrete); steering angle, speed (continuous) |
Known vs unknown | Does the agent (or designer) know the rules or physics of the world? | Ludo (known); a new video game (unknown) |
§3.2 Classifying real environments
Slide L04-S16
Environment | Observable | Agents | Deterministic | Episodic | Static | Discrete |
|---|---|---|---|---|---|---|
Crossword puzzle | Fully | Single | Deterministic | Sequential | Static | Discrete |
Chess with a clock | Fully | Multi | Deterministic | Sequential | Semi | Discrete |
Ludo | Fully | Multi | Stochastic | Sequential | Static | Discrete |
Taxi driving | Partially | Multi | Stochastic | Sequential | Dynamic | Continuous |
Medical diagnosis | Partially | Single | Stochastic | Sequential | Dynamic | Continuous |
X-ray image analysis | Fully | Single | Deterministic | Episodic | Semi | Continuous |
Careful: chess is deterministic - there is no chance. Not knowing what the opponent will do makes it multi-agent, not stochastic. Ludo is stochastic because of the dice. |
4. Five Types of Agents
§4.1 Simple reflex and model-based reflex agents
Slide L04-S18 to S19
A simple reflex agent chooses its action from the current percept only, using condition-action rules (IF condition THEN action), e.g. an automatic street light: IF it is dark THEN switch on. It works only when the environment is fully observable, and it can repeat useless actions forever because it has no memory.
A model-based reflex agent keeps an internal state - its belief about the parts of the world it cannot see now - and updates it with a model of how the world works. A driver remembers the motorbike seen in the mirror a second ago that is now in the blind spot; a robot vacuum keeps a map of the room. It can handle partially observable environments.
§4.2 Goal-based, utility-based and learning agents
Slide L04-S21 to S23
- Goal-based agent: has goal information and asks 'what will happen if I do action A - will it reach my goal?'. It needs search and planning (Lectures 10-18). Example: Google Maps planning a route to UAF Burewala. Changing the goal does not need new rules.
- Utility-based agent: a utility function gives a score for how good a state is. It handles trade-offs (fast vs cheap vs safe) and uncertainty by choosing the highest expected utility. Example: Careem combining arrival time, fare, rating and safety.
- Learning agent: any type plus the ability to improve. Its four parts are the performance element (chooses actions), critic (gives feedback against a performance standard), learning element (makes improvements) and problem generator (suggests new actions to explore). Example: Gmail spam filter learning from 'Report spam'.
Agent type | Decides using | Memory? | Example | Good for |
|---|---|---|---|---|
Simple reflex | Current percept + IF-THEN rules | No | Street light, thermostat | Fully observable, simple tasks |
Model-based reflex | Internal state + model + rules | Yes | Blind-spot driver, robot vacuum map | Partially observable |
Goal-based | Goals + prediction of results | Yes | Google Maps | Planning tasks |
Utility-based | Utility of outcomes | Yes | Careem driver choice | Trade-offs, uncertainty |
Learning | Any of the above + feedback | Yes | Spam filter, YouTube feed | Unknown or changing worlds |
§4.3 Python demo: reflex vs model-based vacuum
Slide L04-S20
Performance measure: +1 for every clean square after each step, -1 for every move. The program below is the one run in class; the output is its real run. Both agents clean both squares by t = 3. The reflex agent then keeps moving (it cannot remember that the other square is clean); the model-based agent remembers and rests (NoOp).
# Vacuum world: 2 squares A and B. Score: +1 for each clean square after every step, -1 for every move. def reflex_agent(percept): location, status = percept if status == "Dirty": return "Suck" return "Right" if location == "A" else "Left" def make_model_agent(): model = {"A": None, "B": None} # internal state: what I believe about each square def agent(percept): location, status = percept model[location] = status if status == "Dirty": model[location] = "Clean" return "Suck" if model["A"] == model["B"] == "Clean": return "NoOp" # everything clean: do nothing, save energy return "Right" if location == "A" else "Left" return agent def run(agent, world, location="A", steps=6): world, score = dict(world), 0 for t in range(1, steps + 1): action = agent((location, world[location])) if action == "Suck": world[location] = "Clean" elif action in ("Right", "Left"): location = "B" if action == "Right" else "A" score -= 1 score += sum(v == "Clean" for v in world.values()) print(f"t={t} {action:<5} at {location} A={world['A']:<5} B={world['B']:<5} score={score}") return score start = {"A": "Dirty", "B": "Dirty"} print("Simple reflex agent:") r = run(reflex_agent, start) print("Model-based agent:") m = run(make_model_agent(), start) print("Final scores -> reflex:", r, "| model-based:", m) |
Simple reflex agent: t=1 Suck at A A=Clean B=Dirty score=1 t=2 Right at B A=Clean B=Dirty score=1 t=3 Suck at B A=Clean B=Clean score=3 t=4 Left at A A=Clean B=Clean score=4 t=5 Right at B A=Clean B=Clean score=5 t=6 Left at A A=Clean B=Clean score=6 Model-based agent: t=1 Suck at A A=Clean B=Dirty score=1 t=2 Right at B A=Clean B=Dirty score=1 t=3 Suck at B A=Clean B=Clean score=3 t=4 NoOp at B A=Clean B=Clean score=5 t=5 NoOp at B A=Clean B=Clean score=7 t=6 NoOp at B A=Clean B=Clean score=9 Final scores -> reflex: 6 | model-based: 9 |
5. Common Mistakes
Slide L04-S26
Wrong idea | Correct idea |
|---|---|
An agent must be a robot. | Software is also an agent: spam filter, Careem app. |
Rational means perfect or all-knowing. | Rational = best expected action with the information available. |
Performance measure = how the agent should behave. | Measure what you want in the environment (clean floor). |
Chess is stochastic because the opponent is unknown. | Chess is deterministic; the opponent makes it multi-agent. |
A goal-based agent always finds the best route. | Goals are yes/no; the best choice needs utility. |
Word Bank
Word | Simple meaning | Example |
|---|---|---|
Sensor | A part that takes input from the environment | Camera, soil-moisture sensor |
Actuator | A part that acts on the environment | Motor, valve, screen |
Percept | What the agent senses at one moment | One camera frame |
Omniscient | Knowing everything, even the future | No real agent is omniscient |
Autonomy | Acting on its own percepts and learning | A robot vacuum that maps a new room |
Observable | Can be seen by the sensors | Chess board is fully observable |
Stochastic | Involving chance; not certain | Dice, weather |
Episodic | Each decision is separate from the others | Checking each X-ray |
Internal state | What the agent remembers or believes about the world | Map of cleaned areas |
Utility | A score of how good or desirable a state is | Fast + cheap + safe ride |
Summary
- An agent perceives through sensors and acts through actuators; agent = architecture + program.
- A rational agent maximises expected performance given its percepts and knowledge; it is not omniscient.
- Describe every task with PEAS: Performance, Environment, Actuators, Sensors.
- Classify environments: observable, agents, deterministic, episodic, static, discrete, known.
- Agent types: simple reflex → model-based → goal-based → utility-based; any can become a learning agent.
Self-Check Questions
- Define agent, sensor and actuator with one example of a software agent. (see §1.1)
- Differentiate an agent function from an agent program. (see §1.2)
- Why is a rational agent not omniscient? Give an example. (see §2.1)
- Why is 'amount of dirt sucked' a bad performance measure for a vacuum? (see §2.2)
- Write PEAS for a crop-spraying drone. (see §2.3)
- Is Ludo deterministic or stochastic? Fully or partially observable? Explain. (see §3.1-3.2)
- Why is X-ray image analysis episodic but medical diagnosis sequential? (see §3.2)
- Explain the weakness of a simple reflex agent using the vacuum demo output. (see §4.1, §4.3)
- Differentiate goal-based and utility-based agents. (see §4.2)
- Name the four components of a learning agent with the spam-filter example. (see §4.2)
Reading: Russell & Norvig (4e) Chapter 2.