LLM + RoboticsModule 18 · Our own course, for ages 15+

Build a Robot Brain: 12 weeks, from zero to a robot that understands you

You'll build Robo-Helper, a robot in a simulated room that understands "put the red block in the bin", plans a route, notices a cat in the way, and reports honestly how well it works. Every idea starts from first principles, and every lesson ends with real code you write and real-world practice away from the screen.

Start here

People don't learn by reading. They learn by building, getting stuck, and fixing it. So that's how this course works.

How every lesson goes

  1. A real problem. Something the robot can't do yet.
  2. First principles. We build the idea from nothing, with pen-and-paper examples.
  3. Play. A small interactive demo in your browser to poke at the idea.
  4. Homework. You write the real Python code, and check.py marks it.
  5. Real-world challenge. Try the idea on something real: your phone, a ramp, a friend.
  6. Project step. Your code becomes part of Robo-Helper.

What you need to know already

Very basic Python: variables, if, for loops, and functions. If you can write a loop that prints 1 to 10, you're ready. Lesson 0 helps you set up.

Time

12 weeks, one session a week of about 2–4 hours. Weeks 1–7 build the robot and its first brains from zero. Weeks 8–11 are inference engineering: measuring how fast and expensive a brain is, making it faster and cheaper, plugging in a world-class model (Claude) over the internet, and racing every brain on the same tasks. Week 12 teaches the robot what people prefer, and you show off your work.

What you'll have built at the end

##########
#Rr......#      > put the blue block in the bin
#..##....#        Something is in the way at (4, 1)! Stopping and finding another route.
#..#..b..#        The blue block is in the bin.
#....##..#      > where is the blue block
#g.......#        The blue block is in the bin.
#.#....B.#
##########      Test set: 83% passed   (and you'll know exactly why the other 17% failed)

That's real output from the finished project. R = robot, r/b/g = blocks, B = bin.

Progress

Lessons done
0 / 21
Next up

The 12 weeks

One week, one session of 2–4 hours (two short lessons count as one week). Badges show which side of AI each lesson builds. Lesson numbers are just names: follow the weeks.

Week 1 · Moving

Set up, then make the robot move on a grid.

00

Install Python, run the robot world, write two tiny functions. Adds: your first robot script.

01

(x, y), directions as small changes, walls. Adds: the robot can move.

Week 2 · Sensing and learning

Noisy sensors, then the idea behind all AI: shrink the error.

02

Measuring distance, why sensors are never exact, why averaging helps. Adds: a distance sensor.

03

Error, then gradient descent: how every AI learns. Adds: learns its own braking distance.

Week 3 · Neurons and honest tests

One artificial neuron, and how to test it so it's safe.

04

Weights, a sum and a squash: the brick every neural network is built from. Adds: an obstacle detector.

05

Test on data the model never saw. Precision, recall, and choosing a safe threshold. Adds: tested stop decisions.

Week 4 · Planning

Find the shortest route with BFS and A*.

06

Breadth-first search, then A*. Adds: route planning.

Week 5 · Words

Turn words into numbers; understand commands.

08

Tokens and next-word prediction, the core idea behind ChatGPT, put straight to work fixing typos the way a phone keyboard does. Adds: typo-proof commands (typo commands handled right: 6/9 → 9/9).

09

A classifier that counts words, plus "slots" for which object. Adds: understands one action per command (bake-off 5/12).

Week 6 · A neural language model

Build a tiny LLM from zero.

15

Embeddings, softmax and next-word prediction in a real (tiny) neural network that writes whole plans word by word ("put red blue"), asks when you're vague, and learns from 343 examples you generate from templates (never from the bake-off's commands). Stretch: add attention, the idea inside Transformers. Adds: multi-step plans (bake-off 5 → 8/12).

Week 7 · Capstone

Safety checks on AI output, then the whole robot, tested.

10

Structured output, strict checks, and a fallback. Adds: a safety layer so AI output can't break the robot.

11

Put it all together and measure it honestly. Adds: the full robot.

Week 8 · Inference I: measure

Latency, tokens and cost.

16

Latency, p50 vs p95, time to first token, tokens per second, and cost per command. How far does the robot roll while it thinks? Adds: a time budget for every brain.

Week 9 · Inference II: speed up

Caching, quantising, distilling, and LoRA fine-tuning.

17

Caching, quantising weights to 8 and 4 bits, and distilling a teacher into a small student. Adds: a brain that learns new phrasings, then shrinks 8× (bake-off 10/12 in 1.4 KB).

13

Fine-tuning vs a tiny low-rank add-on: fix a big model by training a few numbers. Adds: adapts a factory model to your robot.

Week 10 · A world-class brain

Claude as planner and as agent, over the internet.

18

Claude as the robot's brain: a planner that writes a checked plan, and an agent that uses the robot's tools step by step. You stay in charge. Adds: world-class understanding.

Bonus after week 10 · MCP

Needs Python 3.11+ and one install (pip install mcp).

20

Turn the robot's tools into an MCP server that Claude Desktop, Claude Code and other apps can drive, with safety rules no AI can talk its way past. Adds: a real MCP server.

Week 11 · The bake-off

Every brain, scored and replayed.

19

Every brain, the same 12 tasks, the same cat. Score correctness, speed and cost, then watch the replays side by side. Stretch: use an AI model as the judge, catch it being fooled, and measure it honestly with kappa and error bars (Module 19). Adds: a scorecard showing your robot getting better: 5 → 8 → 10/12.

Week 12 · Teach it, show it

Learn from people's preferences; share your project.

12

Show two routes, ask which is better, learn a reward from the answers, pick what people like. The same recipe that trains chat assistants. Adds: picks the route you'd choose.

14

Show your work, then (optionally) put it on real hardware. Adds: your portfolio piece.

Bonus, any time after week 4

07

Rewards and Q-learning: no map, just practice. Adds: learns a maze by itself.

How the project grows

ROBOT BASICS                                   THE BRAIN (LLM side)
wk1 moving ─┐                                  wk5 words ── commands ─┐
wk2 sensors, braking ─┤                        wk6 tiny neural LM ────┤
wk3 neuron, safe tests ┼─► wk7 Robo-Helper ◄── wk7 safety checks ─────┘
wk4 A* planning ─┘          (the capstone)
                                │
             INFERENCE ENGINEERING: running brains well
             wk8  measure: latency, tokens, cost
             wk9  speed up: cache, quantise, distil, LoRA
             wk10 world-class brain: Claude planner + agent
             wk11 bake-off: every brain, same tasks ──► wk12 preferences + show it off

Your robot's report card

Every lesson has to make Robo-Helper better at something you can measure. Run python3 progress.py from project/ after any lesson. It runs your code, and rows you haven't built yet say "not built yet". This is the real output with every lesson finished (--solutions):

week  lesson      ability             measured
1     01          moves               follows the commands EEEESSS from (1, 1) to (5, 3)
2     02          senses              distance sensor error 0.50 -> 0.12 by averaging 16 readings
2     03          learns              learned its braking distance (error 0.33 -> 0.0004); safe speed with 1 m free: 0.9 m/s
3     04-05       decides to stop     stop detector catches 97% of obstacles on new data (precision 84%)
4     06          finds the way       plans the shortest route to the bin: 11 steps (A* looked at 35 squares)
bonus 07          learns by trying    learns a maze by trial and error: 58 steps on the first try -> 10 on the last
5     08          reads typos         typo commands handled like the correct spelling: 6/9 -> 9/9 with the typo fixer
5     09          understands         understands one action per command: bake-off 5/12
6     15          writes plans        writes whole plans and asks when unsure: bake-off 8/12, new commands 92%
6+    15 stretch  attention           attention: new commands 92% (tiny LM 92%; over 3 seeds 92% vs 89%), bake-off 8/12
7     10          stays safe          checks every AI answer before moving: 5 broken answers caught, 3 good ones let through
7     11          capstone            the whole robot, tested end to end: 83% of test commands done right
8     16          speed               knows how long it thinks: tiny LM p95 0.3 ms = 0.3 mm rolled while thinking
9     17          faster, smarter     learns new phrasings from a teacher: bake-off 10/12; 4-bit copy 10/12 at 1411 bytes (was 11288)
9     13          adapts (LoRA)       fixes its leaky sensor with LoRA: danger error 0.067 -> 0.000, training 12 numbers
10    18          world-class brain   … needs recordings (see brains.py record)
11    19          bake-off            the same 12 tasks for every brain: words 5/12, tinylm 8/12, distilled 10/12
12    12          what people like    picks the route people prefer: wall brushes 10 -> 8, turns 3 -> 1 (same 11 steps)
bonus 20          MCP                 any AI app can drive it over MCP, and the server's own rules hold

The brain you build climbs the bake-off week by week, on the same 12 tasks it never trained on: word counter 5/12 (week 5), plan-writing tiny LM 8/12 (week 6), plus a teacher's labels 10/12 (week 9), and still 10/12 squeezed into 1.4 KB. The two tasks every small brain still fails need the robot to see the room ("the block that's closest to the bin") or to know that "the room" means every block. That's the job for Claude in week 10.

Honest caveat: 12 tasks is few. The jump from 5/12 to 8/12 is "too close to call" with error bars; 5/12 to 10/12 is a real difference (week 11 shows how to tell).

Bonus: Module 21 pairs ten interactive maths explainers (Kalman, Bayes, A*, Fourier…) with bonus_robot_math.py, which uses each one on Robo-Helper and measures the difference.

Getting the code

  1. The project lives in topics/18-robot-brain/project/ inside this course folder. Copy that whole project folder somewhere you like.
  2. Read project/README.md.
  3. Your files are in project/homework/. Answers are in project/solutions/, but peek only after a real try.
  4. Check any lesson with python3 check.py 3 (from inside project/).

For parents and teachers