1. Objectives
- Explain function approximation: a network that outputs a NUMBER instead of a class.
- Train MLPRegressor on a curve (y = x², y = sin x) and compare it with a straight line (LinearRegression).
- Measure a regression model with R² and mean squared error (MSE).
- Find the best fertiliser dose from a learned yield curve and choose the hidden layer size.
2. Word Bank
Word | Simple meaning | Real-world example |
|---|---|---|
Function approximation | Learning a rule y = f(x) from examples, so we can predict y for new x | Yield from fertiliser dose |
Regression | Predicting a number (not a class) | Wheat yield in maunds per acre |
MLPRegressor | scikit-learn's multilayer network for numbers | MLPRegressor(hidden_layer_sizes=(4,)) |
MSE | Mean squared error: average of (true - predicted)²; lower is better | MSE of 2.5 |
R² (R-squared) | How much of the variation the model explains: 1 = perfect, 0 = no better than the average | R² = 0.82 |
Optimum | The best value (here the dose that gives the highest yield) | About 70 kg nitrogen per acre |
Maund | A local unit of weight: 1 maund = 40 kg | 40 maunds per acre |
3. Background
From classes to numbers
In P02-P03 the networks answered YES/NO questions (rust or no rust). Many AI problems need a NUMBER: tomorrow's electricity units, the price of cotton, the yield of a field. Learning such a number from examples is called function approximation (or regression). A multilayer network can approximate almost any smooth curve if it has enough hidden neurons - this is why neural networks are called universal approximators.
Real-world link: urea and wheat
Wheat farmers around Burewala add nitrogen (urea). A little nitrogen raises the yield; too much wastes money and can even lower the yield (the crop falls over - lodging). So yield against dose is a CURVE that rises and then falls. A straight line cannot show the best dose; a network can. wheat_nitrogen.csv has 41 farms: nitrogen in kg per acre and yield in maunds per acre.
MLPRegressor in one line
MLPRegressor(hidden_layer_sizes=(4,), activation='tanh', solver='lbfgs', max_iter=5000, random_state=1). It is used exactly like MLPClassifier, but y holds numbers. score() returns R², not accuracy.
Data files for this lab
wheat_nitrogen.csv
farm,nitrogen_kg,yield_maund F01,0,20.8 F02,2.5,23.8 F03,5,22.4 F04,7.5,27.2 F05,10,29.3 F06,12.5,28.1 F07,15,30.6 F08,17.5,31.5 F09,20,31.6 F10,22.5,31.9 ... (41 data rows in total - copy the full file from the lab folder) |
4. Time Plan (50 minutes)
Minutes | Activity | Part |
|---|---|---|
0-5 | Background, word bank and the idea of function approximation | 1-3 |
5-10 | Guided warm-up: a network learns y = x² | Part A |
10-13 | Predict the output | Part B |
13-17 | Bug hunt | Part C |
17-47 | Graded tasks 1-3 | Part D |
47-50 | Wrap-up; post-lab questions set as homework | Part E |
Part A - Guided Warm-up (type, run, observe)
A1. A straight line vs a network on y = x²
61 points of the curve y = x² between -3 and 3. A straight line (LinearRegression) cannot bend; a network with one hidden layer of 20 tanh neurons can.
import numpy as np from sklearn.linear_model import LinearRegression from sklearn.neural_network import MLPRegressor x = np.linspace(-3, 3, 61).reshape(-1, 1) # 61 rows, 1 column y = (x ** 2).ravel() # target numbers line = LinearRegression().fit(x, y) net = MLPRegressor(hidden_layer_sizes=(20,), activation="tanh", solver="lbfgs", max_iter=2000, random_state=1).fit(x, y) print("x = -2, 0, 2 -> line:", line.predict([[-2], [0], [2]]).round(2)) print("x = -2, 0, 2 -> net :", net.predict([[-2], [0], [2]]).round(2)) print("R2 line:", round(line.score(x, y), 3), "| R2 net:", round(net.score(x, y), 3)) |
Output x = -2, 0, 2 -> line: [3.1 3.1 3.1] x = -2, 0, 2 -> net : [4. 0. 4.] R2 line: 0.0 | R2 net: 1.0 |
A2. Look at the wheat data
Always plot the data first. The points rise, level off near 65-75 kg and then fall a little.
import pandas as pd import matplotlib.pyplot as plt df = pd.read_csv("wheat_nitrogen.csv") print(df.shape, "| yield from", df["yield_maund"].min(), "to", df["yield_maund"].max(), "maunds/acre") plt.figure(figsize=(6, 3.2)) plt.scatter(df["nitrogen_kg"], df["yield_maund"], s=18) plt.xlabel("Nitrogen (kg per acre)"); plt.ylabel("Yield (maunds per acre)") plt.title("Wheat yield vs nitrogen (41 farms)"); plt.grid(alpha=0.3) plt.tight_layout(); plt.savefig("wheat.png", dpi=100) print("Saved wheat.png") |
Output (41, 3) | yield from 20.8 to 45.1 maunds/acre Saved wheat.png |
Part B - Predict the Output
Write your prediction BEFORE running the code. Then run it and correct yourself in a different colour.
B1.
from sklearn.metrics import mean_squared_error print(mean_squared_error([3, 5], [2, 7])) |
My prediction |
B2.
from sklearn.metrics import r2_score print(r2_score([10, 20, 30], [10, 20, 30]), r2_score([10, 20, 30], [20, 20, 20])) |
My prediction |
B3.
import numpy as np from sklearn.neural_network import MLPRegressor x = np.array([[0], [1], [2], [3]]); y = np.array([0, 1, 4, 9]) m = MLPRegressor(hidden_layer_sizes=(5,), solver="lbfgs", max_iter=2000, random_state=0).fit(x, y) print(m.predict([[1], [2], [5]]).shape, [w.shape for w in m.coefs_]) |
My prediction |
Part C - Bug Hunt
Each program has ONE bug. Some crash with an error message; others run but give a wrong answer (silent logic error - the dangerous kind). Find the bug, explain it and write the fix.
C1.
import numpy as np from sklearn.neural_network import MLPRegressor nitrogen = np.array([0, 25, 50, 75, 100]) yield_m = np.array([22, 34, 40, 41, 38]) m = MLPRegressor(solver="lbfgs", random_state=1).fit(nitrogen, yield_m) |
Bug, why it happens, and the fixed line(s) |
C2.
import numpy as np from sklearn.neural_network import MLPClassifier x = np.linspace(0, 3, 20).reshape(-1, 1) y = x.ravel() ** 2 # numbers like 0.0249, 0.0997, ... m = MLPClassifier(solver="lbfgs", random_state=1).fit(x, y) |
Bug, why it happens, and the fixed line(s) |
C3.
import numpy as np from sklearn.neural_network import MLPRegressor x = np.linspace(-3, 3, 61).reshape(-1, 1); y = (x ** 2).ravel() net = MLPRegressor(hidden_layer_sizes=(20,), activation="tanh", solver="lbfgs", max_iter=5, random_state=1).fit(x, y) print("R2:", round(net.score(x, y), 3)) |
It runs and prints R2: 0.403 |
Bug, why it happens, and the fixed line(s) |
Part D - Graded Tasks (10 marks)
Task | Level | Title | Marks |
|---|---|---|---|
T1 | Easy | Approximate the sine curve | 3 |
T2 | Medium | Wheat: straight line vs network, and the best dose | 3 |
T3 | Challenge | How many hidden neurons? | 4 |
Total | 10 |
Task 1 (Easy, 3 marks): Approximate the sine curve
Make 50 points of y = sin(x) for x from 0 to 2π (np.linspace). Split them 75/25 with train_test_split(test_size=0.25, random_state=42). Train MLPRegressor(hidden_layer_sizes=(20,), activation='tanh', solver='lbfgs', max_iter=2000, random_state=1). Print the test R² and the prediction for x = π/2 (the true value is 1).
Sample run (your output must match):
Output test R2: 1.0 sin(pi/2) predicted: 0.993 (true 1.0) |
Your code (or attach the file) |
Task 2 (Medium, 3 marks): Wheat: straight line vs network, and the best dose
Load wheat_nitrogen.csv (X = nitrogen_kg as a 1-column table, y = yield_maund) and split it with test_size=0.25, random_state=42. Train LinearRegression and a pipeline make_pipeline(StandardScaler(), MLPRegressor(hidden_layer_sizes=(4,), activation='tanh', solver='lbfgs', max_iter=5000, random_state=1)). Print the test R² of both. Then use the network to predict the yield for every dose 0, 1, ..., 100 kg and print the best dose and its predicted yield.
Sample run (your output must match):
Output test R2 straight line: 0.066 test R2 network (4,) : 0.821 best dose: 73 kg/acre -> predicted yield 42.7 maunds/acre |
Your code (or attach the file) |
Task 3 (Challenge, 4 marks): How many hidden neurons?
With the same split and pipeline, try hidden_layer_sizes (1,), (4,) and (40,). For each print the training R², the test R² and the best dose (0-100 kg). Plot the data and the three learned curves on one figure (save task3.png). Write which size you would give the farmer and why.
Sample run (your output must match):
Output h=1 train R2 0.926 test R2 0.745 best dose 100 kg h=4 train R2 0.952 test R2 0.821 best dose 73 kg h=40 train R2 1.000 test R2 0.558 best dose 59 kg Saved task3.png |
Your code (or attach the file) |
Part E - Post-Lab Questions (finish as homework if time runs out)
E1. What is function approximation? How is it different from the classification you did in P03?
Your answer |
E2. Why could the straight line not find the best nitrogen dose?
Your answer |
E3. What do R² = 1 and R² = 0 mean?
Your answer |
Marking Rubric (applied to Part D)
Criterion | Full | Partial | None |
|---|---|---|---|
Code runs without errors | Runs fully | Small fixable error | Does not run |
Correct data, split and model settings | All correct | One slip | Several wrong |
Results printed and plot labelled | All present | Some missing | None |
Explanation (under/overfitting, best dose) | Clear and correct | Partly correct | Missing |
Submission
- Make a folder RollNo_P04a with task1.py, task2.py, task3.py (or one notebook P04a.ipynb), wheat_nitrogen.csv and task3.png.
- Upload the zipped folder to the LMS / Google Classroom by the end of the day.
- Copy Task 2 and Task 3 code and output into your Lab Book (Practical 04a page).
- Use random_state exactly as given so your numbers match the sample runs.